dedupe: read in larger chunks at the time