Stream byte array into chunks

Akka Akka Streams & Alpakka
1

Hey guys,

I have a quite big byte array which I want to stream, but in chunks of n bytes.
I want to get it as Stream of ByteString in the end.

Maybe something like

Source<ByteString,NotUsed> source = ByteString.fromArray(byteArray,chunks)

I’m looking for a fast solution to chunk it.
Is there maybe an existing solution?

Thank you
Sigurd

2

I used now a custom iterator:

@RequiredArgsConstructor
public class ChunkedByteStringIterator implements Iterator<ByteString> {

  private final byte[] bytes;
  private final int chunks;
  private int offset = 0;

  @Override
  public boolean hasNext() {
    return bytes != null && bytes.length > offset;
  }

  @Override
  public ByteString next() {
    int currentOffset = offset;
    offset += chunks;
    if (currentOffset + chunks >= bytes.length) {
      return ByteString.fromArrayUnsafe(bytes, currentOffset, bytes.length - currentOffset);
    }
    return ByteString.fromArrayUnsafe(bytes, currentOffset, chunks);
  }
}

The given byte array is effectively immutable.

in a stream, it can be used like that:

Source.fromIterator(() -> new ChunkedByteStringIterator(..)).via(anyBusiness())

What do you think about that?

3

Would sth like this work for you?

object ByteArrayChunker extends App {
  implicit val system: ActorSystem = ActorSystem()

//  val sourceFileName = "63MB.pdf"
//  val sourceFilePath = s"src/main/resources/$sourceFileName"
//  val fileInputStream = new FileInputStream(sourceFilePath)

  val byteArrayInputStream = new ByteArrayInputStream(Array(192, 168, 10, 20).map(_.toByte));

  val source: Source[ByteString, Future[IOResult]] = StreamConverters.fromInputStream(() => byteArrayInputStream, chunkSize = 10 * 1024)
  source.map(each => println(each)).runWith(Sink.ignore)
}

Regards
Paul

4

Thank you for this nice solution.
Its very short and easy.

5

Very good post, thank you. I am surprised to find your website.