There are a few collisions in the production archive. I suspect these
are due to poorly modified caches, and tracking the source(s) of each
group will make it easier to determine which cache is probably
problematic.
This change also has the benefit of removing a lot of the hacky source
name/description merging logic.
Signed-off-by: Graham <gpe@openrs2.org>
This is consistent with processMasterIndex and processIndex,
particularly now that the processGroup logic is slightly more
complicated.
Signed-off-by: Graham <gpe@openrs2.org>
This makes us behave like a standard client that only keeps a single
copy of each group in its cache. This ensures we can at least detect
(crc32, version) collisions for a particular group, rather than silently
skipping colliding cached groups.
A disadvantage is that more bandwidth usage is required, especially if
the download is interrupted.
Signed-off-by: Graham <gpe@openrs2.org>
As key validation has to uncompress the data anyway to confirm the key
is valid, it seems silly to uncompress twice given everywhere we
performed key validation immediately uncompressed the container
afterwards.
Signed-off-by: Graham <gpe@openrs2.org>
There's no real use for these yet, but they might be useful with NXT
caches.
We don't need a compressed_length column because it's easy to determine
the length of a BYTEA column within the database.
Signed-off-by: Graham <gpe@openrs2.org>
This fixes the downloader getting stuck if there's a particularly large
file, such that we've read all the current inbound data but haven't sent
a new request yet as we still have 200 in-flight.
Signed-off-by: Graham <gpe@openrs2.org>