1505 Commits

Author SHA1 Message Date
Hans-Christoph Steiner
dd14b9e315 choose random mirror for each package/APK download
This spreads downloads across all available mirrors randomly.  This could
definitely be improved, like choosing the fastest or nearest mirror, or
only .onion addresses on Tor.  This will improve the current situation and
should reduce the load on f-droid.org a lot.

fdroidclient#1696
2019-02-20 13:39:26 +01:00
Hans-Christoph Steiner
89422e9c8f clarify get mirrors method: Repo.getRandomMirror() 2019-02-20 13:39:26 +01:00
Hans-Christoph Steiner
cf9a6b851d RepoAdapter: code cleanup
Remove unused code and simplify to only present args that are used. This is
remnants from:

fdroidclient#490
fdroidclient#606
fdroidclient!295
fdroidclient!242
2019-02-20 13:39:26 +01:00
Hans-Christoph Steiner
afe6de94a0 handle Apache and Nginx ETags when checking if index is current
fdroid/fdroidclient#1708
2019-02-20 13:39:22 +01:00
Hans-Christoph Steiner
9522012fe1 use built-in constants for HTTP status codes 2019-02-19 11:17:09 +01:00
Hans-Christoph Steiner
1587505389 Merge branch 'mirror-tweaks' into 'master'
mirror tweaks

Closes #1702

See merge request fdroid/fdroidclient!789
2019-02-15 13:24:09 +00:00
Conny Duck
074f72f58a fix activity transitions by limiting view rebinding 2019-02-15 07:28:57 +00:00
Hans-Christoph Steiner
57b587dfe3 Some users never use WiFi, check for that state on first run
Lots of people complain that the graphics aren't being downloaded.  That's
because they never use F-Droid while on WiFi and the default prefs do not
allow downloading graphics while on Data.  This sets the preference to
allow downloading graphics while on Data if only Data is enabled, and not
WiFi, when the user first starts F-Droid.

closes #1592
2019-02-14 15:33:57 +01:00
Hans-Christoph Steiner
aebf7efc10 add canonical URL as first item in ordered mirror list
This makes it display nicely in RepoDetails, and is natural, since it is
the canonical URL.  This also maintains the order as received from the
mirror entries in the index file.
2019-02-14 15:33:57 +01:00
Hans-Christoph Steiner
421270ad5f handle implied READ_EXTERNAL_STORAGE permissions
Having _WRITE_EXTERNAL_STORAGE_ will implied _READ_EXTERNAL_STORAGE_:
https://developer.android.com/reference/android/Manifest.permission#READ_EXTERNAL_STORAGE

closes #1702
2019-02-14 15:33:57 +01:00
Hans-Christoph Steiner
66ac4bcec3 also set Read Timeout with mirror logic timeout value
The Read Timeout makes a SocketTimeoutException be thrown if the timeout
expires before data is available for reading from the connection's
returned InputStream.  This should help the client switch to a new mirror
when the current mirror is too slow or overloaded.
2019-02-14 15:33:31 +01:00
Hans-Christoph Steiner
71884c16b1 clarify mirror timeout logic with constants
This should make the timeout logic clearer, without changing the logic at
all.  This does increase the timeouts, with the second pass using 1 minute
instead of 30 seconds, and the third pass using 10 minutes instead of 1
minute.  Since this often or usually runs in the background, it should
allow some pretty long timeouts in the worst case.
2019-02-14 15:33:31 +01:00
Hans-Christoph Steiner
f064e33de9 disable all compression when downloading on < android-19
Compression seems to just give stacktraces:

     HttpDownloaderTest  I  URL: https://en.wikipedia.org/wiki/Index.html
             TestRunner  I  failed: downloadUninterruptedTest(org.fdroid.fdroid.net.HttpDownloaderTest)
                         I  ----- begin exception -----
                         I  java.io.EOFException
                         I      at java.util.zip.GZIPInputStream.readFully(GZIPInputStream.java:206)
                         I      at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:98)
                         I      at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:81)
                         I      at libcore.net.http.HttpEngine.initContentStream(HttpEngine.java:541)
                         I      at libcore.net.http.HttpEngine.readResponse(HttpEngine.java:844)
                         I      at libcore.net.http.HttpURLConnectionImpl.getResponse(HttpURLConnectionImpl.java:283)
                         I      at libcore.net.http.HttpURLConnectionImpl.getHeaderField(HttpURLConnectionImpl.java:139)
                         I      at libcore.net.http.HttpsURLConnectionImpl.getHeaderField(HttpsURLConnectionImpl.java:246)
                         I      at org.fdroid.fdroid.net.HttpDownloader.download(HttpDownloader.java:111)
                         I      at org.fdroid.fdroid.net.HttpDownloaderTest.downloadUninterruptedTest(HttpDownloaderTest.java:74)
                         I      at java.lang.reflect.Method.invokeNative(Native Method)
                         I      at java.lang.reflect.Method.invoke(Method.java:511)
                         I      at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
                         I      at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
                         I      at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
                         I      at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
                         I      at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
                         I      at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
                         I      at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
                         I      at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
                         I      at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
                         I      at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
                         I      at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
                         I      at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
                         I      at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
                         I      at org.junit.runners.Suite.runChild(Suite.java:128)
                         I      at org.junit.runners.Suite.runChild(Suite.java:27)
                         I      at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
                         I      at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
                         I      at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
                         I      at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
                         I      at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
                         I      at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
                         I      at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
                         I      at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
                         I      at android.support.test.internal.runner.TestExecutor.execute(TestExecutor.java:56)
                         I      at android.support.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:384)
                         I      at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1661)
2019-01-07 17:33:36 +01:00
Hans-Christoph Steiner
7bc7fa9288 rename LoggingQuery.query() to rawQuery() to make audits easier
rawQuery() is much more dangerous, so this method should have the
same name as the method it is actually calling.

https://f-droid.org/docs/Second_Audit_Report/#f47--fdroidclient-raw-sql-query-executions
2019-01-03 14:52:32 +01:00
Hans-Christoph Steiner
1deec1c9b3 sanitize all packageNames from the index
This is insurance to make sure that packageNames are not abused for
exploiting F-Droid.  The database queries already use SQL Prepared
Statements, but who know what else might be exploitable.

fdroid/fdroidclient#1588
2019-01-03 14:52:32 +01:00
Hans-Christoph Steiner
26c1ef3033 move RepoXMLHandler to org.fdroid.fdroid.data
This should be lumped with the classes it uses.
2019-01-03 14:52:32 +01:00
Hans-Christoph Steiner
9c8cc20a80 validate all data in repo push requests
This should get us closer towards not having to trust the server.

fdroid/fdroidclient#1588

https://stackoverflow.com/questions/5205339/regular-expression-matching-fully-qualified-class-names/5205467
2019-01-03 14:52:32 +01:00
Hans-Christoph Steiner
a4e80383f9 when index updates fail, try to get localized exception message 2019-01-02 23:23:28 +01:00
Hans-Christoph Steiner
823ddcaca8 use separate titles for Updates pref and Updates tab
https://gitlab.com/fdroid/fdroidclient/merge_requests/773#note_127475129
2019-01-02 23:23:28 +01:00
Hans-Christoph Steiner
079cf2e903 enable lint HardwareIds as error
Keep PRNGFixes as it is since it is security sensitive, standardized
code from Google.  While F-Droid never wants to do anything with
hardware IDs at all, this code uses the Build.SERIAL as a seed for the
random number generator, so it is safe privacy-wise.
2019-01-02 15:40:38 +01:00
Hans-Christoph Steiner
09abc0734e format code in PRNGFixes 2019-01-02 15:40:38 +01:00
Hans-Christoph Steiner
71920f6082 use Cols.ROW_ID/Cols._ID for all "rowid"/"_id" strings 2019-01-02 15:40:27 +01:00
Hans-Christoph Steiner
bce35ef865 remove unused methods in database code to make auditing easier
These were flagged as potential vulnerabilities.
2019-01-02 15:40:27 +01:00
Hans-Christoph Steiner
e4537a4271 Merge branch 'clean-up-sdcard-swap' into 'master'
Clean up sdcard swap

See merge request fdroid/fdroidclient!775
2018-12-25 22:32:23 +00:00
Peter Serwylo
da9eba94eb Remove unused parameter from app provider, after updating icon query 2018-12-22 19:39:03 +11:00
Marcus Hoffmann
4595517aba AppProvider: fix IconUpdateQuery
The query was trying to figure out some thing about suggestedVercode
which shouldn't at all be necessary for setting the iconUrl.
The index already contains the icon pointing to the suggested version by
that repository, so we just take that regardless.
2018-12-22 09:40:16 +11:00
Marcus Hoffmann
43e5ab7eb2 fix typoes in comments 2018-12-22 09:40:16 +11:00
Hans-Christoph Steiner
0a306a4df2 after requesting Storage permissions, start SDCard scan 2018-12-21 23:06:10 +01:00
Hans-Christoph Steiner
20ebc00e5e improved log warning about TargetSdkVersion mismatch 2018-12-21 23:03:08 +01:00
Hans-Christoph Steiner
ae0c712a3e remove animation from BottomBar to make more text fit into labels
For many languages, there are unavoidable long words needed for the labels
on the button bar, for example, the standard word for Settings can be up to
15 characters long:
https://gitlab.com/fdroid/fdroidclient/issues/1569#note_126469088

The BottomBar was scaling the active one up, and sizing all the fields based
on that size.  This removes that animation, and sets all tabs to always have
the same text size.  That makes it possible to make the spacing tighter.
This also sets the text truncating mode to "middle" which sticks an elipsis
in the middle of the truncated word and shows the start and end.

closes #1569
closes !756
2018-12-21 14:27:57 +01:00
Hans-Christoph Steiner
59befbd355 use Android constants for common URI schemes
This makes the code easier to trace.
2018-12-21 14:27:57 +01:00
Hans-Christoph Steiner
f95af36140 allow repos on removable storage to work without any internet 2018-12-21 00:06:15 +01:00
Hans-Christoph Steiner
69e2ca4283 keep F-Droid visible after adding removeable storage mirror 2018-12-21 00:06:15 +01:00
Hans-Christoph Steiner
90c3baf5af scanning WiFi/Bluetooth in android-23 requires location permission
This requires that admin#65 is fixed, otherwise every F-Droid update will
require Unknown Sources with Privileged Extension.

https://developer.android.com/about/versions/marshmallow/android-6.0-changes.html#behavior-hardware-id
https://stackoverflow.com/a/44200390

closes #656
2018-12-21 00:06:13 +01:00
Hans-Christoph Steiner
1d1f489d85 handle repo Intents for mirrors that are already enabled
This adds a new IntentService to pre-process Intents that request a
new repo is added.  Right now, this only handles Intents that come
from the new storage scanners.

This also adds a new case to the AddRepo UI logic to cover when an
incoming Intent is for a mirror that is already included in an enabled
repo.  In that case, the user is show the Repo Details screen for the
repo that includes that mirror.  This is done is a hacky way right now
since the only path through is to click the button.  So this clicks
the button in code.
2018-12-21 00:04:34 +01:00
Hans-Christoph Steiner
c4b0955c96 add preference to disable removable storage scanning 2018-12-21 00:04:34 +01:00
Hans-Christoph Steiner
f9bc219073 SDCardScannerService for using repos from SD Cards
Creates an IntentService subclass for scanning removable "external
storage" for F-Droid package repos, e.g. SD Cards. This is intented to
support sharable package repos, so it ignores non-removable storage,
like the fake emulated sdcard from devices with only built-in storage.
This method will only ever allow for reading repos, never writing.  It
also will not work for removeable storage devices plugged in via USB,
since do not show up as "External Storage"

* https://stackoverflow.com/a/40201333
* https://commonsware.com/blog/2017/11/14/storage-situation-external-storage.html

closes #1377
2018-12-21 00:04:34 +01:00
Hans-Christoph Steiner
1571e28f68 support swapping with removable storage on android-21+
This uses the new Storage Access Framework, which was required for
accessing files on the SD Card starting in android-19.  But the API
was really limited until android-21, and not really complete until
android-23 or even android-26.  So the levels of usability will vary a
lot based on how new the version of Android is.
2018-12-21 00:03:39 +01:00
Hans-Christoph Steiner
ac1a5e0ad8 ensure the canonical repo URL is always included in mirrors list
The mirror logic assumes that it has a mirrors list with at least once
valid entry in it.  In the index format as defined by `fdroid update`,
there is always at least one valid URL: the canonical URL.  That also
means if there is only one item in the mirrors list, there are no
other URLs to try.

The initial state of the repos in the database also include the canonical
URL in the mirrors list so the mirror logic works on the first index
update.  That makes it possible to do the first index update via SD Card
or USB OTG drive.
2018-12-20 22:58:13 +01:00
Hans-Christoph Steiner
f7049a3295 display versionCode in expanded Versions list entries
closes #1326
2018-12-20 13:14:18 +01:00
dkanada
28863cf88f fix #1642 2018-12-18 08:55:32 +00:00
Hans-Christoph Steiner
0e6b4acabf
rename RepoUpdater to IndexUpdater 2018-12-17 17:17:28 +01:00
Hans-Christoph Steiner
148d1cdc8a rename AppDetails2 to AppDetailsActivity
This also moves it into org.fdroid.fdroid.views
2018-12-17 17:16:51 +01:00
dkanada
2b512d96c5 remove html method from changelog section 2018-12-14 02:48:22 +09:00
Hans-Christoph Steiner
551b577bca Merge branch 'badge' into 'master'
use new design for other badge instances

See merge request fdroid/fdroidclient!754
2018-12-12 20:17:47 +00:00
darken
1ea4d0a330 Improve error messages, prevent duplicate information display.
Fixes #1144
2018-12-12 14:08:11 +01:00
dkanada
568ef56247 fix merge issues and tweak the header layouts a bit more 2018-12-09 16:37:18 +09:00
dkanada
24a4da9d89 some layout modifications to Updates and AppDetails headers 2018-12-09 16:28:45 +09:00
Jan-Christoph Borchardt
09a7ec3fd3 Change inconsistent 'Upgrade' to 'Update'
Signed-off-by: Jan-Christoph Borchardt <hey@jancborchardt.net>
2018-12-06 18:01:08 +00:00
Jan-Christoph Borchardt
5f660501e3 Change technical 'Run' to more understandable 'Open'
Signed-off-by: Jan-Christoph Borchardt <hey@jancborchardt.net>
2018-12-06 18:01:08 +00:00