Graham
f0cd4afcf0
Add sorting and filtering to the caches table
...
Signed-off-by: Graham <gpe@openrs2.org>
3 years ago
Graham
e1a650aec4
Update introduction text
...
We now download RuneScape 3 caches automatically.
Signed-off-by: Graham <gpe@openrs2.org>
3 years ago
Graham
3848a89f68
Ensure a directory is created for the ARCHIVESET archive
...
Signed-off-by: Graham <gpe@openrs2.org>
3 years ago
Graham
36f5efa1ad
Serve flat file caches as .tar.gz files instead of .zip files
...
Streaming .tar.gz files requires less memory, as we don't need to
remember metadata about each file for the end of directory record.
Signed-off-by: Graham <gpe@openrs2.org>
3 years ago
Graham
cbb2a90388
Disable .dat2/.idx link if the cache is too big for a DiskStore
...
Signed-off-by: Graham <gpe@openrs2.org>
3 years ago
Graham
a52955fe4b
Switch to ktor's CIO backend
...
It seems to deal with backpressure better than the Netty backend.
Signed-off-by: Graham <gpe@openrs2.org>
3 years ago
Graham
0ec7736764
Log the number of XTEA keys imported
...
This makes it a bit easier to determine if the keys are parsed
correctly.
Signed-off-by: Graham <gpe@openrs2.org>
3 years ago
Graham
b3d6b112d2
Add Logback configuration file
...
There are two main reasons for this change: by default, logback uses
stdout (!) and not stderr. This caused problems in programs like
packclass/unpackclass, where the packclass or JAR files were mixed in
with Netty's debug logging.
Secondly, the debug logging from lots of third-party libraries was
annoying (as it made it difficult to pick out OpenRS2-specific log
messages), so I've disabled lots of it.
Signed-off-by: Graham <gpe@openrs2.org>
3 years ago
Graham
7915698cda
Reduce CacheExporter BATCH_SIZE
...
I think this is causing OOMs.
Signed-off-by: Graham <gpe@openrs2.org>
3 years ago
Graham
821cb732fd
Remove signature verification from the cache downloader
...
The keys seem to change in each release now, which is going to be too
much of a pain to keep up with.
Signed-off-by: Graham <gpe@openrs2.org>
3 years ago
Graham
0781e23874
Add update links to each master index
...
Flenarn provided a spreadsheet mapping our master indexes to the
corresponding update.
Signed-off-by: Graham <gpe@openrs2.org>
3 years ago
Graham
6f02ab2f65
Add NXT cache downloader
...
I'm still not particularly happy with this: if the JS5 download
finishes before HTTP, it'll time out and kill the whole process.
Similarly, because it takes so long to import the indexes and as we
can't fetch groups in parallel with that, it can often time out early
during the process.
In the long term, I think I am going to try and move most of the logic
outside of the Netty threads and communicate between threads with queues
or channels. This would also allow us to run multiple JS5 clients in
parallel.
The code also needs some tidying up, particularly constants in the
Js5ChannelHandler constructors.
Signed-off-by: Graham <gpe@openrs2.org>
3 years ago
Graham
21ed41c307
Use applet viewer config to find the hostname of a JS5 server
...
Signed-off-by: Graham <gpe@openrs2.org>
3 years ago
Graham
11a43242b9
Add applet viewer config parser
...
Signed-off-by: Graham <gpe@openrs2.org>
3 years ago
Graham
637e311a67
Fix SELECT DISTINCT query
...
Signed-off-by: Graham <gpe@openrs2.org>
3 years ago
Graham
fc6e739879
Add e prefix to the name generator
...
Signed-off-by: Graham <gpe@openrs2.org>
3 years ago
Graham
d925b68db3
Remove duplicate keys from valid.json
...
Signed-off-by: Graham <gpe@openrs2.org>
3 years ago
Graham
2c43930db2
Move Js5Archive and Js5ConfigGroup to cache-550
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
01b50e33f7
Fix resolve_group() if the version number is greater than 65535
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
a3cca0e6c3
Add Protocol constants
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
ce741279b4
Add read-only Js5Index.{Group,File} types
...
These will be used by the high-level cache API, where we don't want to
expose mutable versions of the group/file types as that would allow the
index/cache to get out of sync.
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
1c0bf7529c
Update Bootstrap
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
b37e4d0d0b
Use alternatives for most methods deprecated in Kotlin 1.5
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
cf8110e5a7
Fix possible off-by-one error in KeyBruteForcer
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
9e8d1fca1c
Use type-safe project accessors
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
6498d39306
Add Gradle 7 version catalog
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
3293454c17
Add support for importing keys in binary format
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
09961589e3
Fix map square coordinate text
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
ba60fecb50
Add support for downloading caches in flat file format
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
e0a2df889b
Add more information to the individual cache pages
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
d89c45940b
Infer the number of bytes to skip automatically in the material decoder
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
1f9e7798e7
Improve size estimation footnote
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
193af138fe
Use non-breaking space in the size column
...
This has the convenient side effect of us no longer needing to exclude
the byteunits dependency from minimisation.
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
9be1a54b76
Add estimated cache size column
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
ee567cc76f
Treat empty loc groups as valid
...
I've decided to reverse my decision in
e3e0094b43
.
If you treat empty loc groups as valid, some OSRS caches do have 100%
valid keys. Being able to pick these out in the list of caches will be
useful.
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
40d6ccbb61
Add ent's output to the keys page
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
221252895c
Check exit code of ent
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
593f46d9fd
Add percentages to the keys page
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
6c728d758c
Add total number of encrypted/validated groups to the keys page
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
6cf9eb5043
Set source name for original caches to 'Jagex'
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
f1c9fed36f
Rename 'Name' column to 'Source(s)'
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
4337020b6b
Add endpoint for exporting all keys in bulk
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
93ee863e20
Add initial support for plotting a map of valid/invalid XTEA keys
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
da3cc57b3f
Add link to download keys in the text format
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
f6961a50f1
Add cache size in bytes to master_index_stats
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
8bc25e7e47
Set compression lewel to -9
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
aad469c05d
Generate XTEA key zip reproducibly
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
c4f807d38f
Add X-Forwarded-For support
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
2d136c89e9
Add zipped text version of the XTEA key endpoint
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
a02eb169f4
Fix NPE in TextKeyReader if the input is invalid
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago