Byte Range support - performance

I have been using getFromBrowseableDirectories which is simple to use and includes support for byte ranges. The test files include some quite large MP4 files for which a browser will use ranges to skip to a position in the playback. An example request:

2018-04-14T09:30:56.496Z GET /P1070390.MP4 206 Partial Content 2.2ms range(bytes=62554112-176800745) [28.2MiB 569ms 23.2MiB/s]

Here the time from request to returning the HttpResponse object is 2.2ms, BUT it is another 569ms before the first packet of the entity data is returned. This is the time taken to read and discard the first 62MB of the file. Actually surprisingly good, but this is running on my dev machine with an SSD and more memory available than is likely if used in production.

So my next task is to produce a file source which produces range results without simply discarding the unwanted data. Does anyone already have such a source?

I am planning to use the NIO.2 AsynchronousFileChannel as it avoids some issues with the older FileChannel api. On the other hand it does want its own thread pool.

Mark

Looked a bit at the file directives and it’s not obvious how this could easily be optimised (that doesn’t mean it can’t though)

If you have decided to try to create your own custom directive supporting ranges you may be able to use the FileIO.fromPath() factory that accepts a startPosition parameter rather than do skipping in the stream.

There’s some notes on the performance various underlying options for reading files in these PRs: https://github.com/akka/akka/pull/23217#issuecomment-314675056 https://github.com/akka/akka/pull/21801 that may also be interesting.

Sync vs Async file channels is a tricky problem. If you have lots of access to the same file, then with sync io you need a separate channel for each - there is implicit synchronization that otherwise degrades performance, in particular on Windows.
So sync is faster provided you can use a separate channel for each request. If you want to share channels across multiple requests (video is a case where this might be likely), then you ought to use async.
In any case it is important to test on all supported platforms as performance can differ significantly for reasons that are not at all obvious.

I think I will start with a version using FileIO and a startPosition and later attempt an async version.

Thanks for your interest and suggestions.
Mark