The Event class is no longer needed once there is specific ProgressListener
instances for each type of progress update. The sourceUrl serves as the
unique ID, like with DownloaderService and InstallManagerService.
fixes#633https://gitlab.com/fdroid/fdroidclient/issues/633
This allows the install process to have consistent data, even if the index
database changes while an APK is making its way through this process.
This also provides a set of packageNames to be easily queried.
Standardizing on Apk as the internal data type means that most of the data
that is needed for the whole lifecycle of a given APK going through this
process will be available regardless of the database changes. Once App
instances are also included, then all of the data should be available
separately from the database. This is important to support parallel
operation. The index could be updated and an app could disappear while an
APK of that app is being downloaded. In that case, it should not show
blank notifications.
Also, in AppDetail, the Apk instance is completely loaded from the db, so
there should not be any nulls on the essential bits like packageName and
download URL.
This keeps the IDs standard throughout the code: either urlString when it
should be a String, or urlString.hashCode() when it should be an int. It
also follows the naming convention in DownloaderService helper methods,
e.g. getIntentFilter(). "create" to me doesn't necessarily mean also "get"
Using @NonNull or @Nullable is fine when it is actually useful, like the
compiler can catch errors, but it also adds a lot of noise when reading the
code. For example, @NonNull here will just make people avoid thinking.
Context can never be null anywhere in Android, that's a given throughout
the Android API. And in this code, urlString is the unique ID used
throughout the process, so if its ever null, nothing works.
This keeps DownloaderService tightly focused on downloading, and makes it a
lot easier to manage Notifications since InstallManagerService's lifecycle
lasts as long as the Notifications, unlike DownloaderService.
DownloaderService is structured to be as simple as possible, and as tightly
matched to the downloading lifecycle as possible, with a single queue for
all requests to avoid downloads competing for bandwidth. This does not
represent the possibilities of the whole install process. For example,
downloading can happen in parallel with checking the cache, and if an APK
is fully cached, there is no need for it to go through the DownloaderService
queue.
This also lays the groundwork towards simplifying DownloaderService even
more, by moving the Notification handling to InstallManagerService. That
will provide a single place to manage all aspects of the Notifications that
has a lifecycle that is longer than the Notifications, unlike an Activity
or DownloaderService.
Fixes#624.
The `AppDetails` activity was not correctly asking for the active
download url string when being resumed. This change recalculates the
value when being resumed now.
Added assertions that both apkName and repoAddress need to be populated
in order to call `getUrl()`. Also verified that this is the case for all
usages of this method, which it should be. All `Apk` objects which currently
have `getUrl()` called on them are loaded using the `ApkProvider.Helper.findById()`
method without specifying which columns to load (which defaults to all).
Addresses a bug found in MR !273 whereby removing `stopForeground` results
in a persistent "Downloading ..." notification even though it was cancelled.
In the process of doing this, it also addresses / Fixes#621 by ensuring
that all downloads of apks are done in a foreground service, regardless
of the preference used for foreground updater service updating.
DownloaderServiceTest is just a stub of a test that doesn't really test
anything yet. It is now causing a NullPointerException, so its a problem:
java.lang.NullPointerException
at org.fdroid.fdroid.net.DownloaderService.notifyDownloadComplete(DownloaderService.java:313)
at org.fdroid.fdroid.net.DownloaderService.handleIntent(DownloaderService.java:287)
at org.fdroid.fdroid.net.DownloaderService$ServiceHandler.handleMessage(DownloaderService.java:104)
at android.os.Handler.dispatchMessage(Handler.java:99)
at android.os.Looper.loop(Looper.java:137)
at android.os.HandlerThread.run(HandlerThread.java:60)
This method provides the exact same results as the underlying method it
uses, intent.getStringExtra()
This was added in 0163d6efa6013181c2e6554760e5fa6e67a6daf9
There is already a method for reproducibly generating an int based on
a the contents of a String, thanks to @pserwylo for finding it! So
urlString.hashCode() is used as the ID for Notifications and the
DownloaderService queue (msg.what). This entirely replaces the
QUEUE_WHATS hack. Both requestCode and msg.what just need to be
unique int, and that value can always be generated by the urlString.
This also fixes a bug preventing removing correct URL from Downloader queue.
b66810944fec802aa119c0e5ec8b7875930a2c22 made a change that breaks removing
the correct item from the queue. The `what` value must be fetched based on
urlString and fed to serviceHandler.removeMessages(what). The commit made
it use the `what` value from the last item that was queued. Those will
often be different things.
This also removes all stopForeground(true) calls except for the one in
onDestroy(). The nature of an IntentService, which DownloaderService
is basically a copy of, is to quit running once it is no longer
processing Intents. That is also the time to remove the notification.
Before, these were not being reliably canceled before the final COMPLETE or
INTERRUPTED notification went out. This moves closing the progress Timer
to a finally block after the Timer is setup, which should hopefully
guarantee the progress reports are always stopped.
This prevents any more progress reports from being sent, in case there is
one pending anywhere. downloaderProgressListener needs to be volatile to
ensure that the two threads are seeing the same values.
This was an omission on my part when putting together the DownloaderService
in !248
Having the notification as its own Service is overkill and really only
serves to increase complexity. The notification stuff should not take much
time or resources at all.
Test downloads with actual files, not dynamically generated things.
Testing with the progress reports is really hard with multiple URLs, so
just test progress with a single URL for now, and multiple URLs can still
be tested without the progress check.
fixes#650https://gitlab.com/fdroid/fdroidclient/issues/650
Mirror the check already being done in the other fragment. As reported
via ACRA:
ANDROID_VERSION=4.4.4
APP_VERSION_NAME=0.100-alpha6
STACK_TRACE=java.lang.NullPointerException
at org.fdroid.fdroid.AppDetails$AppDetailsHeaderFragment.updateViews(AppDetails.java:1524)
at org.fdroid.fdroid.AppDetails$AppDetailsHeaderFragment.updateViews(AppDetails.java:1519)
at org.fdroid.fdroid.AppDetails.refreshHeader(AppDetails.java:624)
at org.fdroid.fdroid.AppDetails.onAppChanged(AppDetails.java:549)
at org.fdroid.fdroid.AppDetails.access$000(AppDetails.java:100)
at org.fdroid.fdroid.AppDetails$AppObserver.onChange(AppDetails.java:141)
at android.database.ContentObserver$NotificationRunnable.run(ContentObserver.java:180)
at android.os.Handler.handleCallback(Handler.java:733)
at android.os.Handler.dispatchMessage(Handler.java:95)
at android.os.Looper.loop(Looper.java:136)
at android.app.ActivityThread.main(ActivityThread.java:5146)
at java.lang.reflect.Method.invokeNative(Native Method)
at java.lang.reflect.Method.invoke(Method.java:515)
at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:732)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:566)
at dalvik.system.NativeStart.main(Native Method)
This was very probably introduced with the new CleanCacheService stuff.
Reported via ACRA:
ANDROID_VERSION=6.0.1
APP_VERSION_NAME=0.100-alpha6
STACK_TRACE=java.lang.NullPointerException: Attempt to get length of null array
at org.fdroid.fdroid.Utils.clearOldFiles(Utils.java:349)
at org.fdroid.fdroid.Utils.clearOldFiles(Utils.java:351)
at org.fdroid.fdroid.Utils.clearOldFiles(Utils.java:351)
at org.fdroid.fdroid.CleanCacheService.onHandleIntent(CleanCacheService.java:54)
at android.app.IntentService$ServiceHandler.handleMessage(IntentService.java:66)
at android.os.Handler.dispatchMessage(Handler.java:102)
at android.os.Looper.loop(Looper.java:148)
at android.os.HandlerThread.run(HandlerThread.java:61)
There is nothing to clear if the directory could not be obtained for
some reason, so this seems like a reasonable fix. Anything is better
than a crash anyway.
See the following crash reported via ACRA:
java.lang.NullPointerException
at org.fdroid.fdroid.net.DownloaderService.getNotificationTitle(DownloaderService.java:188)
at org.fdroid.fdroid.net.DownloaderService.createNotification(DownloaderService.java:167)
at org.fdroid.fdroid.net.DownloaderService.handleIntent(DownloaderService.java:274)
at org.fdroid.fdroid.net.DownloaderService$ServiceHandler.handleMessage(DownloaderService.java:107)
at android.os.Handler.dispatchMessage(Handler.java:102)
at android.os.Looper.loop(Looper.java:136)
at android.os.HandlerThread.run(HandlerThread.java:61)
I'm not sure what the source of the null App was (could be that we
couldn't access the DB for some reason, or perhaps that somehow the app
wasn't in it anymore). In any case, it doesn't hurt to double check
here.
If the app is null, simply fall back to the title without the app name.
For some reason, even when showing the entire description some words at
the end of lines and paragraphs still get cut off. This results in
little parts of the description being completely inaccessible.
Ellipsizing is just cosmetic, so ditch it on 2.X altogether. I tested
and proved that this fixes the issue with an android-10 x86 emulator,
after reproducing the bug.
Keep the ellipsizing on newer versions, as it doesn't misbehave on
those.
Fixes#510.
The only remaining error was ClipboardCompat, which was unnecessarily
exposing three top-level classes. Make the two implementation classes be
nested, private and static.
Installer UI updates and lint fixes
This updates UI classes of the installer to reflect the changes of AOSP's installer and fixes some lint errors.
* No more differentiation between personal and device permissions (didn't even notice that there was a differentiation here...)
* Icon for "other" permission group
I carefully merged only the changes from AOSP that are related to us and decided against maintaining different versions of the AppSecurityPermissions.
There will be more changes and discussions to the installer UI coming in the next days.
See merge request !277
Translators:
Adrià García-Alzórriz Spanish
Alberto Moshpirit Spanish
Danial Behzadi Persian
Enol P Asturian
Kristjan Räts Estonian
Marcelo Santana Portuguese (Brazil)
Mohamad Hasan Al Banna Indonesian
Verdulo Esperanto
Verdulo Polish
CleanCacheService
This creates `CleanCacheService` to do all of the cache clean up at the lowest possible priority. It also adds a preference to set how long to keep cached APKs.
See merge request !260
Updater speed improvements during "Saving Application Details"
I've been able to reproduce #324 OutOfMemory errors on an emulator with 12MiB of heap space. This branch did _not_ have an OOM error when updating with F-Droid and F-Droid Archive repos enabled. They successfully update without problem.
Fixes#45.
The "Saving Application Details" stage of repository updating is where each apk has its suggested version calculated, icon URLs calculated, etc. These all require [correlated subqueries](https://en.wikipedia.org/wiki/Correlated_subquery) resulting in a full scan of the apk table for each row in the app table. This takes in the order of 25 seconds on my Moto X 2nd Gen.
This branch improves this process by doing the queries in an [sqlite in-memory database](https://sqlite.org/inmemorydb.html), with the results transferred to the database on disk when done. The time required drops from 25 seconds to ~0.5 seconds on my device.
*Note:* I was hoping this would also improve the "Processing ..." Part of the udpater, given it was inserting into an in memory table instead of on disk. If it did have any effect, it was negligible though, so that part is still likely slower than it could be. Each 50 apps (and their associated apks) takes between 150ms and 500ms on my Moto X.
*Secondary Note:* When creating the in memory database, I create indexes for some columns as per before. This technically should slow down the inserts we do, however in practice they had almost no effect. As such, I've left the index creation there because it is required for the correlated subqueries to not suck.
See merge request !269
Security-sensitive code should not be changed unless there is a good reason
to do so. It is too easy to introduce bugs. This change does not address
an issue, so I'm reverting it. See comment in javadoc header for the class.
This reverts commit 2074718391c2c17a974218bc6565cce2dc05407e for just the
RepoUpdater.java file.
This schedules CleanCacheService to run regularly, and delete files older
than the value set in the new "Keep cached apps" preference. It auto-
migrates the old "Cache packages" pref to the new one. The default cache
time for people who did not have "Cache packages" enabled is one day.
This moves the cache file deletion to a dedicated IntentService that runs
at the lowest possible priority. The cache cleanup does not need to happen
with any kind of priority, so it shouldn't delay the app start or take any
resources away from foreground processes.
This also changes the logic around the "Cache packages" preference. The
downloader always saves APKs, then if "Cache packages" is disabled, those
APKs are deleted when they are older than an hour.
This also simplifies Utils.deleteFiles() since the endswith arg is no
longer needed.
* if there is a file there, remove it
The paths are all from the system, so are safe. No SanitizedFile is needed.
Plus, this method was not checking if the original and sanitized versions
where different, and instead just creating the sanitized version. I worry
that could cause odd bugs.
By putting these comments into javadoc, they are directly describing the
code where it is, and there are many tools in IDEs for searching, viewing,
sorting, etc. javadoc comments. Plain comments do not have those tools.
