There are still some gaps but I want to get this committed and possibly
deployed before doing further work.
Remaining items include:
- Mach-O support
- New engine loader ArtifactLink support
- Post-668 client support
- FunOrb support
Signed-off-by: Graham <gpe@openrs2.org>
If the checksum changes in the future then the flyway_schema_history
table needs to be adjusted, which is awkward.
Signed-off-by: Graham <gpe@openrs2.org>
Bit of a corner case, but if we ever encounter an index with a checksum
and version of 0 that resolves then this will ensure the statistics are
consistent between the overall cache and the index row.
Signed-off-by: Graham <gpe@openrs2.org>
These replace the whirlpool, group count and total uncompressed length
columns, which were kind of useless - in particular:
* The group count is also represented with the new stats columns.
* The total uncompressed length overflows, as some indexes are now
larger than 2 gigabytes. One of the new stats columns contains the
compressed size of each archive, which isn't too different.
Signed-off-by: Graham <gpe@openrs2.org>
This will allow us to import FunOrb caches without worrying about the
risk of collisions with the main set of RuneScape caches.
Signed-off-by: Graham <gpe@openrs2.org>
Keys are now initially imported into a key_queue table, which is never
locked exclusively - allowing the API endpoint to function while the
brute forcer is running. The brute forcer moves all pending keys in the
queue to the keys table before running the actual brute forcing.
Signed-off-by: Graham <gpe@openrs2.org>
Streaming .tar.gz files requires less memory, as we don't need to
remember metadata about each file for the end of directory record.
Signed-off-by: Graham <gpe@openrs2.org>
I'm still not particularly happy with this: if the JS5 download
finishes before HTTP, it'll time out and kill the whole process.
Similarly, because it takes so long to import the indexes and as we
can't fetch groups in parallel with that, it can often time out early
during the process.
In the long term, I think I am going to try and move most of the logic
outside of the Netty threads and communicate between threads with queues
or channels. This would also allow us to run multiple JS5 clients in
parallel.
The code also needs some tidying up, particularly constants in the
Js5ChannelHandler constructors.
Signed-off-by: Graham <gpe@openrs2.org>
