I have no Asterisk. But I understand that Asterisk plays an audio file and after playing is finished, Asterisk terminates the call.
In PhonerLite you can see at the statistics page during a call the length/size of buffered data for receipt. That size would be in theory the only delay and should not exceed 100 milliseconds. Are that 100 ms the missing audio that could not be heard at the end?
Either you append some silence after your "normal" audio data or you insert some short delay before hanging up.

Keep in mind that you need a jitter buffer and sound device needs some buffers too at the remote side. Only because you think you sent all audio and disconnects the call, the remote party may not handled all received audio data. Therefore you should really use that short pause. There are not always softphones in use. If there are calls from other devices via other VoIP providers to your your Asterisk, you have different behavior.