Graham
e8fd432f14
Combine key validation with uncompression
...
As key validation has to uncompress the data anyway to confirm the key
is valid, it seems silly to uncompress twice given everywhere we
performed key validation immediately uncompressed the container
afterwards.
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
91d4e46c89
Rename GroupKey to Key
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
3289af5ddf
Add uncompressed_{length,crc32} columns to the containers table
...
There's no real use for these yet, but they might be useful with NXT
caches.
We don't need a compressed_length column because it's easy to determine
the length of a BYTEA column within the database.
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
b003e02ef4
De-duplicate key validation code in KeyBruteForcer
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
8b1880cf09
Add heading to download links column
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
12e3a0eed2
Remove extraneous newline
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
eae9a3e2f4
Remove master index digest column from the web page
...
It isn't really necessary now we have the download links.
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
619424321f
Add index version consistency check
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
97640774f9
Store index protocol, version and flags in the database
...
Although it isn't necessary, we might as well as it doesn't take up much
extra space and we already store all the properties for all groups and
files.
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
1c061c0f66
Store master index format in the database
...
We can't reliably infer it based on the contents of the container.
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
5d7bd5b5c7
Add support for signed master indexes
...
This commit also changes the way the master index format detection
works, as the previous scheme could not distinguish VERSIONED from
WHIRLPOOL.
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
7b6c0cf1fd
Use block comments for multi-line comments
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
4126abfaad
Add support for unversioned master indexes
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
4154e4fdb4
Store non-truncated versions in the archiving service if available
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
cbeb9a3a67
Add mapsquare key for compatibility with Polar's XTEA key files
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
5cfb234cc7
Use Ktor's Content-Disposition builder
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
412d6f4c1f
Add XTEA key export endpoint
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
98dcbedeaf
Sort install() calls in ArchiveModule
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
a5ac7b0b28
Fix heading case
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
4fa67c37fc
Improve code for merging an old and new master index together
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
a19a69b21e
Add name and description to the master_indexes table
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
47127113f4
Add initial archiving service web interface
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
6e9ba80d0c
Rename dev.openrs2.net to org.openrs2.net
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
a3593e9326
Rename master_index_entries to master_index_archives
...
This naming is a bit more consistent with the index_groups and
index_files tables.
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
4ba78e26b9
Create .idx files for empty indexes
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
273f7eddf8
Use a CTE to reduce duplication in CacheExporter
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
9834dccfdd
Add indexes to the groups table
...
This will reduce the impact of checksum/version collisions, as a
collision would have to happen between two indexes of the same archive
rather than any two indexes.
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
723bd42a8e
Simplify encrypted flag in the Container class
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
6a87159f30
Add lengths and uncompressed checksum to index_groups
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
77e227c93d
Store game, build and timestamp in the master index table
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
f3ea8e71fe
Rename version to build
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
689c315bf3
Track current build number automatically
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
fdd0f118f2
Add logging to Js5ChannelHandler
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
d67a5e690f
Add request() method
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
80a8889106
Call read() if there is any existing in-flight request
...
This fixes the downloader getting stuck if there's a particularly large
file, such that we've read all the current inbound data but haven't sent
a new request yet as we still have 200 in-flight.
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
435a8b4eff
Validate checksums in the JS5 client
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
0f939935cd
Skip flush/read calls if no writes are made
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
e0a9bc3ca6
Rename suspend to awaitSuspend
...
Ideally I'd like to just call this await, but it conflicts with an
existing method.
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
29ac56fe1e
Replace flushRequests() with channelReadComplete()
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
f87d89fe7c
Add initial command for downloading the cache from a JS5 server
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
95108126f1
Skip addMasterIndex/addIndex if the index already exists
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
fff63285fe
Re-combine CacheImporter and ContainerImporter
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
4e90bd76b5
Remove the cache and cache_index tables
...
These effectively duplicate the master index tables, but in a less
flexible manner - as they don't support importing a master index where
some of the indexes are missing.
This commit also combines MasterIndexImporter with CacheImporter, to
make it easier to re-use code.
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
03b30145a9
Replace INT with INTEGER for consistency
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
30b605d719
Store master indexes in the database
...
I have a small collection of these from when I ran a service that polled
the JS5 server for the master index back in 2009. We'll probably be able
to find some in old Rune-Server threads too.
They'll be useful for verifying caches with an unclear provenance.
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
19b6893681
Fix V1__init.sql package
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
f2400117fd
Fix typo (are -> area)
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
fc1af29bdf
Make BATCH_SIZE const
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
f143eef142
Move Kotlin files from src/{main,test}/java to src/{main,test}/kotlin
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
ce5b76ef0f
Rename dev.openrs2 package to org.openrs2
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
