Changed the helpful comments to a Javadoc comment, as tooling such as editors
will be more likely to make use of it like that.
Renamed to emphasise that only trailing new lines are stripped.
Added a basic test for this function to ensure it only strips trailing,
and also that it does actually strip trailing slashes.
Doesn't do anything except create an app with no versions,
no donate links, anything like that, and ensure that the adapter
is able to create the view holders for each resulting item.
In the future we can beef this up to check more exotic conditions,
such as calling `updateItems(App)` with different apps, each
with different combinations of versions, donation links, permissions,
etc.
During development of a new feature, I noticed a bug occuring only after using
the new feature for several days. This was because the metadata only infrequently
changes in ways which cause certain code paths to be hit. By having the f-droid.org
metadata from several days apart in the test suite, it allows for testing more
of these cases. In the future, even later versions of the metadata can be added
to ensure that we can update happily from old to new metadata.
Renamed the `Schema.AppMetadata.Categories.CATEGORIES` constant into the
`Schema.AppMetadata.ForWriting.Categories.CATEGORIES` constant. This is
to make it very clear that it is not for reading from the database.
When updating existing apps or inserting new apps, instead of splatting
a comma separated list into a single sqlite3 column, we now put it into
several rows in the `CatJoinTable`. This is done after deleting existing
categories, to make sure that if the app has been removed from a category,
then this is reflected during the update.
The previous category tests only checked that certain categories
would indeed find their way into the database if certain app metadata
is saved. It didn't check the other direction, using these categories
in queries.
<uses-permissions/> tags can have min and max SDK to take effect. This is
not supported currently, and it necessary especially with the privileged
installer so it can properly represent the permissions that an APK is
requesting.
For example:
<uses-permission
android:name="android.permission.MANAGE_ACCOUNTS"
android:maxSdkVersion="22" />
<uses-permission-sdk-23
android:name="android.permission.CAMERA" />
<uses-permission-sdk-23
android:name="android.permission.CALL_PHONE"
android:maxSdkVersion="23" />
It was a little arbitrary to choose this date. However it was when the database
looked quite close to what it looks like now and it is from well over two years
ago. Going into the future, this test may as well always start out at 42 forever
more to ensure that database migrations from that point continue to work for
all future database migrations.
OBB files are used in apps that need more than 100 megs to work well. This
is apps like MAPS.ME or games that put map info, media, etc. into the OBB
file. Also, OBB files provide a mechanism to deliver large data blobs that
do not need to be part of the APK. For example, a game's assets do not
need to change often, so they can be shipped as an OBB, then APK updates do
not need to include all those assets for each update.
https://developer.android.com/google/play/expansion-files.html
Many times in the past, we would ask for an apk based on its package name
and its version code. However multiple repositories provide apks with the
same package name and version code, and such queries would (seemingly)
nondeterministically choose one of these matching apks. This clarifies the
wording in the code around when we explicitly ask for a given apk, and
when we kind of guess which one we want.
Most the time we have an `App` handy, which has a specific repo associated
with it. This allows us to be more specific about requesting `Apk`s.
The times we are kind of guessing is when we rely on the "suggested version
code" of an apk by clicking the misc "Install" or "Upgrade" button in
app details. In the future, we'll need to clear this up so that a more
specific apk is chosen when touching these buttons.
The query which dynamically figured out the preferred metadata based on
repo priority ended up being quite slow (although it did work). On lower
end devices, it has the potential to make F-Droid quite sluggish. By
optimistically precalculating the preferred metadata where possible, we
don't need to ask the question during the usual usage of F-Droid, only
when:
* Repo priorities are changed (there is not currently a UI for this, but
there are tests)
* Repos are enabled/disabled
* Repo updates are performed
Includes:
* One of the functions querying for apps did not correctly specify
the repository the repos came from.
* Fix deletion code which refered to incorrect field.
* Cleanup code style in some places.
The tests are in the .updater packate to make them easier to run as
a suite in Android Studio. Now the package can be right clicked and
ran to run all the tests to do with updating.
The index jar files were updated so as to include info in most
metadata fields (e.g. URLs/descriptions/summary/etc) to show that
that particular part of metadata came from a specific repo. This
will allow more specific tests to show that we can indeed query for
an app with metadata provided by the repo with the highest priority.
Originally, I hoped that the arguments a method took would help enough
to differentiate the intent of that method. This was the case for methods
such as `getContentUri()` and `find()`. However they are a little confusing
to work with, so this change renames a bunch of methods to be more specific.
In addition, it makes some renames from app -> package which will help with
the upcoming change to add a `package` table to the database.
This allows whitelabel versions of apps to specify built-in app repos that
have push requests accepted by default. This is useful for the case where
there is a central manager of the core apps that are installed.
https://gitlab.com/fdroid/fdroidserver/issues/177
This is a step towards supporting easy whitelabeling, using gradle flavors.
This allows the whitelabel version to set the default repos just by making
their own default_repos.xml in app/src/whitelabel/res/values. That one
will then override the built-in F-Droid one.
There are oddities with the way that Android has implemented the network
stack, as compared to OpenJDK or Oracle JDK. So running the tests on the
local JVM, i.e. Robolectric, will not provide good test coverage for real
world use cases.
The code only existed so that it could be used in a test. Subsequently,
a further test was written to test this code (used by the first test).
Since none of the code is actually used in the app, it has been removed.
See #511 for details. This is in prepration for having an even more normalized
`fdroid_package` table. That table will be the authoritative reference of what
"packages" are known about in the client. The "app" table (now thought of as "app metadata") will
be specific to each repository which provides different metadata about that app.
The test was using a `findIgnored` method in `AppProvider`, which only
existed for the purpose of testing. The test has been changed to instead
check for apps which would end up in the "can update" list (which is really
where the "ignored" apps are useful).
Wherever the "package name" of an apk is required, it can be requested by
asking for `Schema.ApkTable.Cols.App.PACKAGE_NAME`. Note the `App` which
indicates that it is in fact pulling this data from the `fdroid_app` table rather
than the `fdroid_apk` table.
When downloading arbitrary URLs using F-Droid (e.g. icons, .apk files, indexes)
then it may be the case that the repo requires authentication. As such, we try
to infer the repository based purely on the URL.
The old code took the basename of the URL, which means remove the last fragment
(e.g. "index.jar") and use the remaining portion of the URL to lookup the repo.
This is broken for many reasons, partly because of the presence of a query string,
partly because there are other things which are not just in the root directory
of the repo (e.g. "/icons/*.png").
This new method iteratively peels off the right most segment of the URLs path,
then looks to see if a repo exists at that address.
Note that this breaks down if you have nested repositories on a server, where
one of the repositories is nested inside a directory that F-Droid knows about,
such as "icons". In such a case, the following repositories:
* https://f-droid.org/repo (requires auth)
* https://f-droid.org/repo/icons (doesn't require auth)
will break down. If requesting something from the repo requiring auth:
* https://f-droid.org/repo/icons/org.fdroid.fdroid.png
Then it will lookup the database and find the repo which lives in "/icons"
and doesn't require auth (or requires a different auth username/password).
Not sure there is a lot that can be done about this without major refactoring.
Such refactoring would require making sure a `Repo` is always given to a downloader
for any HTTP request, and is probably a bit out of scope of this bug.
Also added tests for this behaviour.
Fixes#711.
This is important for the ability to refactor the database for better performance in the future.
See #511 for details.
For those interested in the details, here is a query plan of selecting the "All" category of apps before
this commit:
* `SCAN TABLE fdroid_app USING INDEX app_id`
* `SEARCH TABLE fdroid_apk USING INDEX apk_id (id=?)`
* `SEARCH TABLE fdroid_repo USING INTEGER PRIMARY KEY (rowid=?)`
* `SEARCH TABLE fdroid_installedApp AS installed USING INDEX sqlite_autoindex_fdroid_installedApp_1 (appId=?)`
* `SEARCH TABLE fdroid_apk AS suggestedApk USING INDEX sqlite_autoindex_fdroid_apk_1 (id=? AND vercode=?)`
* `USE TEMP B-TREE FOR ORDER BY`
And here is a query plan of afterwards:
* `SCAN TABLE fdroid_app`
* `SEARCH TABLE fdroid_apk USING INDEX apk_appId (appId=?)`
* `SEARCH TABLE fdroid_repo USING INTEGER PRIMARY KEY (rowid=?)`
* `SEARCH TABLE fdroid_installedApp AS installed USING INDEX sqlite_autoindex_fdroid_installedApp_1 (appId=?)`
* `SEARCH TABLE fdroid_apk AS suggestedApk USING INDEX apk_appId (appId=?)`
* `USE TEMP B-TREE FOR ORDER BY`
The things of note are:
* `SCAN TABLE` doesn't use an index, which means that it is really using the rowid index. Shouldn't behave much differently.
* The second item now uses an integer primary key index rather than a package name index. Should increase search speed marginally which was the goal of this commit. As more apks exist, the speed improvement will also increase.
