The user running the migration might not have superuser permissions.
Using IF NOT EXISTS allows a root user to install the extension
manually, and then the migration will succeed as an unprivileged user.
Signed-off-by: Graham <gpe@openrs2.org>
It isn't solved yet, but once we've committed to v1 of the archiving
service we won't want to edit V1__init.sql ever again to avoid changing
its checksum (Flyway complains if the checksum changes).
Signed-off-by: Graham <gpe@openrs2.org>
Some non-empty loc groups are also unreachable, so I think this was
quite deceptive - e.g. on some OSRS revisions, we'll probably never hit
100% of the keys even if we exclude empty loc groups.
We can include the empty loc flag in the list of missing keys on the
per-cache pages instead.
Signed-off-by: Graham <gpe@openrs2.org>
The previous code over-counted as the use of LEFT JOINs meant candidate
group rows with the incorrect container_id were still included in the
results. Using an IN clause with a subquery allows us to remove those
rows, though it's a bit hacky. (Really I want to be able to use a JOIN
on the right side of a LEFT JOIN to restrict the rows that appear in the
results of the LEFT JOIN, but that doesn't seem to be possible.)
Signed-off-by: Graham <gpe@openrs2.org>
This commit also adds support for populating the whirlpool column, and
ensures version is set to 0 for the ORIGINAL master index format.
Signed-off-by: Graham <gpe@openrs2.org>
I'm not sure if the auto-detection code works: I'm assuming that the new
format was introduced at the same time as the lengths flag in Js5Index,
but I haven't confirmed this.
Signed-off-by: Graham <gpe@openrs2.org>
This makes us behave like a standard client that only keeps a single
copy of each group in its cache. This ensures we can at least detect
(crc32, version) collisions for a particular group, rather than silently
skipping colliding cached groups.
A disadvantage is that more bandwidth usage is required, especially if
the download is interrupted.
Signed-off-by: Graham <gpe@openrs2.org>
The CTE is now declared as NOT MATERIALIZED to ensure Postgres is able
to push the WHERE master_index_id condition inside it.
Signed-off-by: Graham <gpe@openrs2.org>
There's no real use for these yet, but they might be useful with NXT
caches.
We don't need a compressed_length column because it's easy to determine
the length of a BYTEA column within the database.
Signed-off-by: Graham <gpe@openrs2.org>
Although it isn't necessary, we might as well as it doesn't take up much
extra space and we already store all the properties for all groups and
files.
Signed-off-by: Graham <gpe@openrs2.org>
These effectively duplicate the master index tables, but in a less
flexible manner - as they don't support importing a master index where
some of the indexes are missing.
This commit also combines MasterIndexImporter with CacheImporter, to
make it easier to re-use code.
Signed-off-by: Graham <gpe@openrs2.org>
I have a small collection of these from when I ran a service that polled
the JS5 server for the master index back in 2009. We'll probably be able
to find some in old Rune-Server threads too.
They'll be useful for verifying caches with an unclear provenance.
Signed-off-by: Graham <gpe@openrs2.org>
