1869 Commits

Author SHA1 Message Date
Hans-Christoph Steiner
14b4a7e00a cache all downloads based on canonical URL, not download URL
This makes the download cache be shared across all mirrors used to download
rather than having a cache per-mirror.
2019-02-20 13:39:26 +01:00
Hans-Christoph Steiner
dd14b9e315 choose random mirror for each package/APK download
This spreads downloads across all available mirrors randomly.  This could
definitely be improved, like choosing the fastest or nearest mirror, or
only .onion addresses on Tor.  This will improve the current situation and
should reduce the load on f-droid.org a lot.

fdroidclient#1696
2019-02-20 13:39:26 +01:00
Hans-Christoph Steiner
89422e9c8f clarify get mirrors method: Repo.getRandomMirror() 2019-02-20 13:39:26 +01:00
Hans-Christoph Steiner
cf9a6b851d RepoAdapter: code cleanup
Remove unused code and simplify to only present args that are used. This is
remnants from:

fdroidclient#490
fdroidclient#606
fdroidclient!295
fdroidclient!242
2019-02-20 13:39:26 +01:00
Hans-Christoph Steiner
afe6de94a0 handle Apache and Nginx ETags when checking if index is current
fdroid/fdroidclient#1708
2019-02-20 13:39:22 +01:00
Hans-Christoph Steiner
9323ccdfd1 add HTTP Last-Modified header to nearby/swap webserver
This should support the new cache check scheme when using swap repos.
2019-02-19 11:17:09 +01:00
Hans-Christoph Steiner
9522012fe1 use built-in constants for HTTP status codes 2019-02-19 11:17:09 +01:00
Hans-Christoph Steiner
683a990da8 include test string comment to see if Weblate supports them 2019-02-18 23:52:41 +01:00
Hans-Christoph Steiner
b5de4898b1 improve "try it" permissions prompt on Nearby screen
This should make all the translations fit on the screen properly.

closes #1715
2019-02-18 23:52:41 +01:00
Hans-Christoph Steiner
8e8d4a4af6 espresso tests: print RAM info to logcat 2019-02-18 23:52:40 +01:00
Hans-Christoph Steiner
bb597537f2 Translated using Weblate (Chinese (Simplified))
Currently translated at 100.0% (428 of 428 strings)
2019-02-15 14:58:59 +00:00
Hans-Christoph Steiner
1587505389 Merge branch 'mirror-tweaks' into 'master'
mirror tweaks

Closes #1702

See merge request fdroid/fdroidclient!789
2019-02-15 13:24:09 +00:00
Conny Duck
074f72f58a fix activity transitions by limiting view rebinding 2019-02-15 07:28:57 +00:00
Hans-Christoph Steiner
57b587dfe3 Some users never use WiFi, check for that state on first run
Lots of people complain that the graphics aren't being downloaded.  That's
because they never use F-Droid while on WiFi and the default prefs do not
allow downloading graphics while on Data.  This sets the preference to
allow downloading graphics while on Data if only Data is enabled, and not
WiFi, when the user first starts F-Droid.

closes #1592
2019-02-14 15:33:57 +01:00
Hans-Christoph Steiner
aebf7efc10 add canonical URL as first item in ordered mirror list
This makes it display nicely in RepoDetails, and is natural, since it is
the canonical URL.  This also maintains the order as received from the
mirror entries in the index file.
2019-02-14 15:33:57 +01:00
Hans-Christoph Steiner
421270ad5f handle implied READ_EXTERNAL_STORAGE permissions
Having _WRITE_EXTERNAL_STORAGE_ will implied _READ_EXTERNAL_STORAGE_:
https://developer.android.com/reference/android/Manifest.permission#READ_EXTERNAL_STORAGE

closes #1702
2019-02-14 15:33:57 +01:00
Hans-Christoph Steiner
66ac4bcec3 also set Read Timeout with mirror logic timeout value
The Read Timeout makes a SocketTimeoutException be thrown if the timeout
expires before data is available for reading from the connection's
returned InputStream.  This should help the client switch to a new mirror
when the current mirror is too slow or overloaded.
2019-02-14 15:33:31 +01:00
Hans-Christoph Steiner
8f854c2c6e improve source string language for auto-updates pref 2019-02-14 15:33:31 +01:00
Hans-Christoph Steiner
71884c16b1 clarify mirror timeout logic with constants
This should make the timeout logic clearer, without changing the logic at
all.  This does increase the timeouts, with the second pass using 1 minute
instead of 30 seconds, and the third pass using 10 minutes instead of 1
minute.  Since this often or usually runs in the background, it should
allow some pretty long timeouts in the worst case.
2019-02-14 15:33:31 +01:00
Hans-Christoph Steiner
3c185d6d64 enable Espresso tests to auto run, now there are x86 emulators 2019-01-08 20:37:55 +01:00
Hans-Christoph Steiner
22773f3de3 gitlab-ci: force skip Espresso tests on < android-25
It seems that ARM emulators timeout even when just trying to run the
assumeTrue() tests via Espresso.  There needs to be one test still enabled
in the file, otherwise, the run fails with:

org.fdroid.fdroid.MainActivityEspressoTest > initializationError[Nexus_One_API_19(AVD) - 4.4.2] FAILED
        java.lang.Exception: No runnable methods
        at org.junit.runners.BlockJUnit4ClassRunner.validateInstanceMethods(BlockJUnit4ClassRunner.java:191)
2019-01-08 20:36:32 +01:00
Hans-Christoph Steiner
97ad4ddc1f get HttpDownloaderTest working on at least android-17 and above
The two excluded URLs seem to always resolve to IPv6 addresses first, then
fail since there isn't IPv6 connectivity. Donno why, but only on old android
versions, so just skip them there.
2019-01-08 20:35:49 +01:00
Hans-Christoph Steiner
f064e33de9 disable all compression when downloading on < android-19
Compression seems to just give stacktraces:

     HttpDownloaderTest  I  URL: https://en.wikipedia.org/wiki/Index.html
             TestRunner  I  failed: downloadUninterruptedTest(org.fdroid.fdroid.net.HttpDownloaderTest)
                         I  ----- begin exception -----
                         I  java.io.EOFException
                         I      at java.util.zip.GZIPInputStream.readFully(GZIPInputStream.java:206)
                         I      at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:98)
                         I      at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:81)
                         I      at libcore.net.http.HttpEngine.initContentStream(HttpEngine.java:541)
                         I      at libcore.net.http.HttpEngine.readResponse(HttpEngine.java:844)
                         I      at libcore.net.http.HttpURLConnectionImpl.getResponse(HttpURLConnectionImpl.java:283)
                         I      at libcore.net.http.HttpURLConnectionImpl.getHeaderField(HttpURLConnectionImpl.java:139)
                         I      at libcore.net.http.HttpsURLConnectionImpl.getHeaderField(HttpsURLConnectionImpl.java:246)
                         I      at org.fdroid.fdroid.net.HttpDownloader.download(HttpDownloader.java:111)
                         I      at org.fdroid.fdroid.net.HttpDownloaderTest.downloadUninterruptedTest(HttpDownloaderTest.java:74)
                         I      at java.lang.reflect.Method.invokeNative(Native Method)
                         I      at java.lang.reflect.Method.invoke(Method.java:511)
                         I      at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
                         I      at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
                         I      at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
                         I      at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
                         I      at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
                         I      at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
                         I      at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
                         I      at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
                         I      at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
                         I      at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
                         I      at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
                         I      at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
                         I      at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
                         I      at org.junit.runners.Suite.runChild(Suite.java:128)
                         I      at org.junit.runners.Suite.runChild(Suite.java:27)
                         I      at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
                         I      at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
                         I      at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
                         I      at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
                         I      at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
                         I      at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
                         I      at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
                         I      at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
                         I      at android.support.test.internal.runner.TestExecutor.execute(TestExecutor.java:56)
                         I      at android.support.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:384)
                         I      at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1661)
2019-01-07 17:33:36 +01:00
Hans-Christoph Steiner
5ceaa95a34 Weblate 2019-01-04 08:19:32 +00:00
Hans-Christoph Steiner
7bc7fa9288 rename LoggingQuery.query() to rawQuery() to make audits easier
rawQuery() is much more dangerous, so this method should have the
same name as the method it is actually calling.

https://f-droid.org/docs/Second_Audit_Report/#f47--fdroidclient-raw-sql-query-executions
2019-01-03 14:52:32 +01:00
Hans-Christoph Steiner
1deec1c9b3 sanitize all packageNames from the index
This is insurance to make sure that packageNames are not abused for
exploiting F-Droid.  The database queries already use SQL Prepared
Statements, but who know what else might be exploitable.

fdroid/fdroidclient#1588
2019-01-03 14:52:32 +01:00
Hans-Christoph Steiner
26c1ef3033 move RepoXMLHandler to org.fdroid.fdroid.data
This should be lumped with the classes it uses.
2019-01-03 14:52:32 +01:00
Hans-Christoph Steiner
9c8cc20a80 validate all data in repo push requests
This should get us closer towards not having to trust the server.

fdroid/fdroidclient#1588

https://stackoverflow.com/questions/5205339/regular-expression-matching-fully-qualified-class-names/5205467
2019-01-03 14:52:32 +01:00
Hans-Christoph Steiner
9a04ce4332 Merge branch 'fix-screenshot-background' into 'master'
fix screenshot background for all themes in app details

Closes #1618

See merge request fdroid/fdroidclient!782
2019-01-03 13:34:03 +00:00
Hans-Christoph Steiner
a4e80383f9 when index updates fail, try to get localized exception message 2019-01-02 23:23:28 +01:00
Hans-Christoph Steiner
823ddcaca8 use separate titles for Updates pref and Updates tab
https://gitlab.com/fdroid/fdroidclient/merge_requests/773#note_127475129
2019-01-02 23:23:28 +01:00
Conny Duck
e3a24a042d fix screenshot background for all themes in app details 2019-01-02 19:33:59 +01:00
Hans-Christoph Steiner
079cf2e903 enable lint HardwareIds as error
Keep PRNGFixes as it is since it is security sensitive, standardized
code from Google.  While F-Droid never wants to do anything with
hardware IDs at all, this code uses the Build.SERIAL as a seed for the
random number generator, so it is safe privacy-wise.
2019-01-02 15:40:38 +01:00
Hans-Christoph Steiner
09abc0734e format code in PRNGFixes 2019-01-02 15:40:38 +01:00
Hans-Christoph Steiner
cf5b9520fb fix lint "Implied locale in date format" 2019-01-02 15:40:38 +01:00
Hans-Christoph Steiner
178371bf54 fix typos in translations caught by lint 2019-01-02 15:40:38 +01:00
Hans-Christoph Steiner
71920f6082 use Cols.ROW_ID/Cols._ID for all "rowid"/"_id" strings 2019-01-02 15:40:27 +01:00
Hans-Christoph Steiner
bce35ef865 remove unused methods in database code to make auditing easier
These were flagged as potential vulnerabilities.
2019-01-02 15:40:27 +01:00
Hans-Christoph Steiner
945dbdbd3c prevent NPE in Bluetooth swap
ACRA  E  ACRA caught a IllegalStateException for org.fdroid.fdroid.debug
                         E  java.lang.IllegalStateException: Fatal Exception thrown on Scheduler.Worker thread.
                         E      at rx.internal.schedulers.ScheduledAction.run(ScheduledAction.java:62)
                         E      at android.os.Handler.handleCallback(Handler.java:751)
                         E      at android.os.Handler.dispatchMessage(Handler.java:95)
                         E      at android.os.Looper.loop(Looper.java:154)
                         E      at android.app.ActivityThread.main(ActivityThread.java:6128)
                         E      at java.lang.reflect.Method.invoke(Native Method)
                         E      at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:889)
                         E      at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:779)
                         E  Caused by: java.lang.NullPointerException: Attempt to invoke virtual method 'int java.lang.String.hashCode()' on a
                            null object reference
                         E      at org.fdroid.fdroid.localrepo.peers.BonjourPeer.hashCode(BonjourPeer.java:41)
                         E      at sun.misc.Hashing.singleWordWangJenkinsHash(Hashing.java:48)
                         E      at java.util.HashMap.put(HashMap.java:423)
                         E      at java.util.HashSet.add(HashSet.java:217)
                         E      at rx.internal.operators.OperatorDistinct$1.onNext(OperatorDistinct.java:62)
                         E      at rx.internal.operators.OperatorObserveOn$ObserveOnSubscriber.pollQueue(OperatorObserveOn.java:202)
                         E      at rx.internal.operators.OperatorObserveOn$ObserveOnSubscriber$2.call(OperatorObserveOn.java:162)
                         E      at rx.internal.schedulers.ScheduledAction.run(ScheduledAction.java:55)
                         E      ... 7 more
2018-12-26 00:29:19 +01:00
Hans-Christoph Steiner
b264688385 Weblate 2018-12-25 23:05:19 +00:00
Hans-Christoph Steiner
e4537a4271 Merge branch 'clean-up-sdcard-swap' into 'master'
Clean up sdcard swap

See merge request fdroid/fdroidclient!775
2018-12-25 22:32:23 +00:00
Marcus
928042540f Merge branch 'fix_iconquery' into 'master'
Fix iconquery

Closes #1108

See merge request fdroid/fdroidclient!715
2018-12-23 11:06:16 +00:00
Peter Serwylo
da9eba94eb Remove unused parameter from app provider, after updating icon query 2018-12-22 19:39:03 +11:00
Peter Serwylo
d82023b943 Add test for loading app icons from correct repository 2018-12-22 19:34:23 +11:00
Hans-Christoph Steiner
fc474ddf58 Weblate 2018-12-21 23:10:37 +00:00
Marcus Hoffmann
4595517aba AppProvider: fix IconUpdateQuery
The query was trying to figure out some thing about suggestedVercode
which shouldn't at all be necessary for setting the iconUrl.
The index already contains the icon pointing to the suggested version by
that repository, so we just take that regardless.
2018-12-22 09:40:16 +11:00
Marcus Hoffmann
43e5ab7eb2 fix typoes in comments 2018-12-22 09:40:16 +11:00
Hans-Christoph Steiner
0a306a4df2 after requesting Storage permissions, start SDCard scan 2018-12-21 23:06:10 +01:00
Hans-Christoph Steiner
c97424f054 show Toast when scanning an SDCard for repos 2018-12-21 23:06:04 +01:00
Hans-Christoph Steiner
43ee48ac2b only show "SDCards can be used to swap" if an SD Card is present 2018-12-21 23:05:24 +01:00