1124 Commits

Author SHA1 Message Date
Hans-Christoph Steiner
7e7ec966ee improved internet state handling for updates, including metered
This introduces three network states:

1. completely disconnected
2. connected only via metered networks
3. connected via unlimited networks

This allows the update process to use bandwidth better, especially when the
user has enabled the "Only on WiFi" setting.  It also helps prevent silly,
cryptic error messages in the update process is triggered when there isn't
internet available.

I tested this with:

* 4G only, but not set up for internet
* 4G only, with internet
* 4G + WiFi
* WiFi only airplane mode
* no internet at all, full airplane mode

closes #793
closes #774
2016-11-14 16:03:37 +01:00
Hans-Christoph Steiner
e14cb9d16a treat ethernet as WiFi when checking updates
Its really easy to use USB Ethernet devices with ChromeOS and some Android
devices like Android TV.  ChromeOS now supports Android apps.  Since really
the goal is to avoid metered networks, and ethernet is very rarely metered,
this fits in with the user expectations around the preference.  And if it
doesn't, there are very few people using Ethernet with F-Droid right now,
so whatever harm does happen will affect an extremely limited number of
people.
2016-11-14 09:36:45 +01:00
Hans-Christoph Steiner
6545a26e31 set HTTP User Agent to "F-Droid"
First, this is more honest than just using the default since it is saying
what the actual software is.  Second, it protects identity, since the
default User Agent on Android can have a lot of info in it, for example:

"Dalvik/2.1.0 (Linux; U; Android 5.1; XT1039 Build/LPBS23.13-17.3-1)"
2016-11-14 09:36:45 +01:00
Hans-Christoph Steiner
ab1e869ebe use HEAD request when just checking the file size
This code will be changed again when implementing the client-side etag
check #562

closes #777
2016-11-10 20:44:53 +01:00
Hans-Christoph Steiner
096b7132c4 prevent AppDetailsHeaderFragment crash in startProgress()
The real solution would involve figuring out where to handle this in the
right spot in the lifecycle.  Since AppDetails is being totally replaced,
this is just to stop the crashing.

closes #802
2016-11-10 20:44:53 +01:00
Hans-Christoph Steiner
b9dad4bce6 handle dirs and I/O errors when parsing ACTION_PACKAGE_ADDED Intents
InstalledAppProviderService tries to keep a running log of what is actually
installed on the device.  It seems that ApplicationInfo.sourceDir and
related things sometimes returns a dir rather than an APK. So try to find
an APK in that folder.

closes #801
2016-11-10 20:44:53 +01:00
Hans-Christoph Steiner
b852c0dca0 Merge branch 'category-table' into 'master'
Store categories in separate category table

Currently, the category that an app is in is recorded in the database via the `fdroid_app.categories` column, containing a comma separated list of strings. This makes it hard to query. The existing code to get a list of categories was pretty bad as a result. 

This moves to a different data model whereby categories are stored in a separate table. Each repo is free to specify that an app is in arbitray caregories (as with before). This is represented by a join table between categories and app metadata.

The end result is that categories are much more a first class citizen than before, and they will be able to be queried easier - which is important for the new UI.

Note that the categories table need never be emptied, it can keep being appended to. The reason is that if there are no apps in a particular category (represented by no corresponding rows in the join table) then the category will not be shown to the user.

See merge request !409
2016-11-10 10:19:06 +00:00
Hans-Christoph Steiner
0c6ca09a7d Merge branch 'partially-fix-520--performance-when-changing-preference' into 'master'
Improve performance when changing "Unstable Updates" preference

This is only a partial solution to #520. 

It does as I suggested in the issue comments, by doing less work when the preference is checked. The proper solution is an `IntentService` which queues requests to update these details, but that is a (slightly) larger change for the future.

I also noticed it wasn't correctly notifying the UI of the change, so this now notifies the list of apps which can be updated. That was hard to test though, so not sure if it updates the UI correctly or not. It shouldn't do it _incorrectly_, but it may not work. The reason it should work is because the `AppListFragment` (baseclass of `CanUpdateAppsFragment`) uses a cursor loader with the `AppProvider.getCanUpdateUri()` URI. This `CursorLoader` should automatically attach an observer for that URI and requery if required.

See merge request !414
2016-11-10 10:17:03 +00:00
Peter Serwylo
b2c497e5b9 Only update suggested versions, not other unneeded things.
Also, while we are here, lets notify the content observers that the suggested
versions have changed, so that the UI can update in response.
2016-11-10 12:57:49 +11:00
Peter Serwylo
fdc95c071d Rename from 'calc app details' to 'calc all details'
In preperation for a subsequent change to only calculate a subset of details
for performance reasons.
2016-11-10 12:25:55 +11:00
Peter Serwylo
f060efb7ba Remove outdated doc comment and simplify method. 2016-11-10 12:22:37 +11:00
Peter Serwylo
a7a52fbfba Fall back to InstalledAppProvider when trying to identify the apk to uninstall.
Extracted `getInstalledApk()` method so that it could be better documented, and
makes the `uninstallApk()` more consise. It will now throw an `IllegalStateException`
if no apk is found, because as issue #800 shows we will end up with a NPE otherwise.

Fixes issue #800.
2016-11-10 11:16:29 +11:00
Peter Serwylo
9785536910 Use the Query object rather than the SQLiteDatabase#query() method.
This ensures that all of the relevant joins are in place, so that when
the updater asks to `queryPackageName()` then it can assume that we have
already joined onto the `Schema.PackageTable`.
2016-11-10 09:09:43 +11:00
Peter Serwylo
354f0a9b53 Make it explicit that the CATEGORIES column is not for selecting from.
Renamed the `Schema.AppMetadata.Categories.CATEGORIES` constant into the
`Schema.AppMetadata.ForWriting.Categories.CATEGORIES` constant. This is
to make it very clear that it is not for reading from the database.
2016-11-10 08:09:49 +11:00
Peter Serwylo
68f666685f Make category helper functions query new category table 2016-11-10 08:09:49 +11:00
Peter Serwylo
634fe1084a Move category helper functions to CategoryProvider
Don't change anything yet, just move them.
2016-11-10 08:09:49 +11:00
Peter Serwylo
a7a7f77b42 Refactored categories field from column in metadata table to join table.
When updating existing apps or inserting new apps, instead of splatting
a comma separated list into a single sqlite3 column, we now put it into
several rows in the `CatJoinTable`. This is done after deleting existing
categories, to make sure that if the app has been removed from a category,
then this is reflected during the update.
2016-11-10 08:09:49 +11:00
Peter Serwylo
b2d5bcc94a When querying based on category, use join table. 2016-11-10 08:09:49 +11:00
Peter Serwylo
3e3fdd5c07 Added category to keyword search. 2016-11-10 08:09:49 +11:00
Peter Serwylo
8757acca1a Added category provider.
Not used by anything yet.
2016-11-10 08:09:49 +11:00
Peter Serwylo
99f7cab62e Added "category" table and "category to app metadata" join table.
Nothing uses these added tables yet.
2016-11-10 08:09:49 +11:00
Peter Serwylo
bb7cfc14cc Comments for the Schema.PackageTable. 2016-11-10 08:09:49 +11:00
Peter Serwylo
19ca68cb30 Removed unused category view in app details.
It was hidden some time ago, and nobody seems to miss it.
Also, we will be redoing this view soon anyway. In the meantime,
this category stuff is changing and this view should be removed.
2016-11-10 08:09:49 +11:00
Peter Serwylo
d65e72638b Reimplement columnExists using PRAGMA table_info.
For some reason, the existing approach of "select * and then see if the
column of interest is in the results set" didn't work as expected under
tests. Perhaps SQLite is caching the list of columns for the purpose of
`select *` even after running an `alter table add column` query?

Either way, I couldn't figure out why it wasn't working as expected.
This left us with two options:

 * Try to `select columnToCheck` and see if it throws an exception
 * Query columns using `PRAGMA table_info.

The exception thrown when a column doesn't exist is not specific enough
for our code to check that this is the exact exception that occured. It
is not possible to say: `try { ... } catch (SQLiteColumnNotFound e) { ...}`
unfotunately. Also, there is a cost associated with unwinding the stack
to process an exception, which means exceptions probably shouldn't be
used in unexceptional circumstances such as this.

This change instead uses `PRAGMA table_info(tableOfInterest)` and then
iterates over the cursor looking for the relevant column. Even if the
performance is worse than the stack unwinding of an exception, it is
more concise and less hacky.
2016-10-19 06:44:08 +11:00
Peter Serwylo
63a609fbab Moved methods away from top of DBHelper class.
The fact there are arbitrary migrations at the top of the file (between
`onCreate()` and `onUpdate()` makes it harder to scan this file.

This changeset moves these methods verbatim, without changing any of
the method bodies or signatures.
2016-10-18 18:00:36 +11:00
Peter Serwylo
a317877120 Be a little more concise about what to do when running migration for v64.
Before it is not apparant that the migration introduced for v64 is
associated with that particular version. This is because the guard
condition used to bail out from the upgrade is more closely related
to a previous migration.

This is due to a flaw with the desigh of `resetTransient()`, whereby
it always resets the database to the schema _of the current F-Droid
version being run_, not of the tables as they stood at the time of
the particular migration being introduced.

This clarifies the guard condition for v64 by instead explicitly asking
if the columns of interest exist yet in this particular invocation of
`onUpgrade()`.
2016-10-18 18:00:30 +11:00
Peter Serwylo
b72cdff522 Guard against null, and improve logging in ApkVerifier. 2016-10-16 20:28:19 +11:00
Peter Serwylo
0e70495046 Guard against getRunningAppProcesses() returning null 2016-10-13 06:42:49 +11:00
Peter Serwylo
e2e1d3111c Extract code to check for ACRA process into method
Makes it easier to document the code and simplifies FDroidApp#onCreate().
2016-10-13 06:20:06 +11:00
Hans-Christoph Steiner
6f0c9ff88a support extended 'uses-permissions' tags in APKs
<uses-permissions/> tags can have min and max SDK to take effect.  This is
not supported currently, and it necessary especially with the privileged
installer so it can properly represent the permissions that an APK is
requesting.

For example:
<uses-permission
  android:name="android.permission.MANAGE_ACCOUNTS"
  android:maxSdkVersion="22" />
<uses-permission-sdk-23
  android:name="android.permission.CAMERA" />
<uses-permission-sdk-23
  android:name="android.permission.CALL_PHONE"
  android:maxSdkVersion="23" />
2016-10-11 08:44:51 +02:00
Hans-Christoph Steiner
d7022dd498 rename Apk.permissions to requestedPermissions like PackageInfo
android.content.pm.PackageInfo is the Android class for representing data
about an APK/package.  Since Apk.permission is the same thing, we should
use the same name.
2016-10-11 08:44:51 +02:00
Hans-Christoph Steiner
a5a90954bc fix lint UnsafeProtectedBroadcastReceiver
Android won't protect us from other apps sending other Intents to these
receivers, so at least check that the action string matches what its
looking for.  This is based on a lint recommendation.
2016-10-10 20:15:47 +02:00
Hans-Christoph Steiner
15181d47f5 use apply() with all SharedPreferences
if the code does not check the result of commit() it should use apply()
since that runs in the background.
2016-10-10 19:57:55 +02:00
Peter Serwylo
72a88583d6 Only drop fdroid_installedApp if it exists 2016-10-11 00:15:38 +11:00
Peter Serwylo
0d4d160407 Fix migration for DB version 50.
The migration resulted in a query being run which was broken. The query
was broken because it was dynamically generated by Java code. This Java
code resulted in a valid migration when until very recently when the
query was refactored to deal with a new DB structure. Now the query is
no longer suitable to be run against a DB_VERSION 49 database.

To resolve this, the migration now hard codes the query to a string
which is executable when the DB_VERSION is 49.
2016-10-11 00:15:35 +11:00
Peter Serwylo
050d9974b7 Added a test which runs all DB migrations since DB version 42.
It was a little arbitrary to choose this date. However it was when the database
looked quite close to what it looks like now and it is from well over two years
ago. Going into the future, this test may as well always start out at 42 forever
more to ensure that database migrations from that point continue to work for
all future database migrations.
2016-10-10 23:42:05 +11:00
Hans-Christoph Steiner
4b2aec6d08 fix crash loop when upgrading from v0.101
For upgrades from DB version earlier than 63, the whole table is recreated
by resetTransient() in migrateToPackageTable() so the upgrade method for
the OBB tables only needs to run when the database is at exactly version 63

This was mistakenly added to cd9582c9902dd4ac9218acfd69872f3eebcd3d93 when
it was rebased on !375.
2016-10-06 23:22:56 +02:00
Hans-Christoph Steiner
65c4087b05 improved Apk.toString() for easier debugging 2016-10-06 21:06:54 +02:00
Hans-Christoph Steiner
470145e611 remove unused ContentValuesCursor class
This was replaced entirely by making Apk implement Parcelable
2016-10-06 21:06:54 +02:00
Hans-Christoph Steiner
e1a6c931c6 make sure uninstall process has an Apk instance
If a user clicks install, then uninstall on AppDetails, then there was not
yet a chance to refresh the App instance, and therefore app.installedApk
will still be null.  This is really just a workaround for now, because
AppDetails needs a full refactoring.
2016-10-06 21:06:54 +02:00
Hans-Christoph Steiner
b8162a1a91 InstallManagerService.cancel() to handle all cancellation
Now that there are also OBB downloads, there needs to be a central cancel
method provided by InstallManagerService.
2016-10-06 21:06:54 +02:00
Hans-Christoph Steiner
8affa08d11 auto-download and -install any associated OBB files
This implements the APK Extension Files spec for finding, downloading, and
installing OBB files that are extension packs for APKs.

This needs WRITE_EXTERNAL_STORAGE since "installing" OBB files is just
copying them to a specific path on the external storage.

https://developer.android.com/google/play/expansion-files.html
2016-10-06 19:12:01 +02:00
Hans-Christoph Steiner
4c4aef5314 refactor into reusable static method for checking file hashes
This takes the APK file hash checker and turns it into a generic static
utility method for checking that a given file matches a given hash.  This
will be needed as F-Droid handles other file types, like OBB and media.
2016-10-06 18:00:25 +02:00
Hans-Christoph Steiner
a5e6dad9bf allow apps to request OBB download URLs from F-Droid
By sending an Intent to F-Droid, it will reply with the full download URL
to the OBB file, if one exists for the currently installed version of the
requesting app.
2016-10-06 18:00:25 +02:00
Hans-Christoph Steiner
bbac03b4d1 use ApkTable column names when parsing XML
This makes it easier to track the relationship between the index XML and
the database tables where that data is ultimately stored and used. There
are a few mismatches between the XML tag and database column names, so
those are just marked with a comment.

This makes it much easier to find all the spots in the code that need
changing when adding new columns/data to the APK table, like the OBB stuff.
In Android Studio, just Ctrl-Click on any table constant definition, and
then it lists all the places its used.  Any new data will need to be added
in all of those locations.
2016-10-06 18:00:25 +02:00
Hans-Christoph Steiner
cd9582c990 support "APK Extension" files aka .obb for large apps and games
OBB files are used in apps that need more than 100 megs to work well.  This
is apps like MAPS.ME or games that put map info, media, etc. into the OBB
file.  Also, OBB files provide a mechanism to deliver large data blobs that
do not need to be part of the APK.  For example, a game's assets do not
need to change often, so they can be shipped as an OBB, then APK updates do
not need to include all those assets for each update.

https://developer.android.com/google/play/expansion-files.html
2016-10-06 18:00:25 +02:00
Peter Serwylo
e0a1d2384d Appease checkstyle + pmd 2016-10-06 03:03:02 +11:00
Peter Serwylo
d062af0975 Clarify that sometimes we don't know which repos apk we are asking for.
Many times in the past, we would ask for an apk based on its package name
and its version code. However multiple repositories provide apks with the
same package name and version code, and such queries would (seemingly)
nondeterministically choose one of these matching apks. This clarifies the
wording in the code around when we explicitly ask for a given apk, and
when we kind of guess which one we want.

Most the time we have an `App` handy, which has a specific repo associated
with it. This allows us to be more specific about requesting `Apk`s.

The times we are kind of guessing is when we rely on the "suggested version
code" of an apk by clicking the misc "Install" or "Upgrade" button in
app details. In the future, we'll need to clear this up so that a more
specific apk is chosen when touching these buttons.
2016-10-06 03:03:02 +11:00
Peter Serwylo
3a24d21f59 WIP: Making correct apks get found when updating repo. 2016-10-06 03:03:02 +11:00
Peter Serwylo
ab02058ece Precalculate the preferred metadata, rather than always at runtime
The  query which dynamically figured out the preferred metadata based on
repo priority ended up being quite slow (although it did work). On lower
end devices, it has the potential to make F-Droid quite sluggish. By
optimistically precalculating the preferred metadata where possible, we
don't need to ask the question during the usual usage of F-Droid, only
when:
 * Repo priorities are changed (there is not currently a UI for this, but
  there are tests)
 * Repos are enabled/disabled
 * Repo updates are performed
2016-10-06 03:03:02 +11:00