3908 Commits

Author SHA1 Message Date
Peter Serwylo
55c2700d67 WIP: Making correct apks get found when updating repo. 2016-10-25 11:54:41 +03:30
Peter Serwylo
aa96de6730 Moved regression test to appropriate package. 2016-10-25 11:54:41 +03:30
Peter Serwylo
1590145f54 Precalculate the preferred metadata, rather than always at runtime
The  query which dynamically figured out the preferred metadata based on
repo priority ended up being quite slow (although it did work). On lower
end devices, it has the potential to make F-Droid quite sluggish. By
optimistically precalculating the preferred metadata where possible, we
don't need to ask the question during the usual usage of F-Droid, only
when:
 * Repo priorities are changed (there is not currently a UI for this, but
  there are tests)
 * Repos are enabled/disabled
 * Repo updates are performed
2016-10-25 11:54:41 +03:30
Peter Serwylo
7ebf9f050a Clarify some of the database stuff around database providers.
Includes:
 * One of the functions querying for apps did not correctly specify
   the repository the repos came from.
 * Fix deletion code which refered to incorrect field.
 * Cleanup code style in some places.
2016-10-25 11:54:41 +03:30
Peter Serwylo
ad52b92911 Update priorities for default repos to go from 1-4 instead of 10 + 20 2016-10-25 11:54:41 +03:30
Peter Serwylo
d68404767a Cleaning up/commenting AppProvider 2016-10-25 11:54:41 +03:30
Peter Serwylo
54f215090b Clarify a limitation in the current implementation
Two repositories can (and always could) end up with the same exact .apk file.
If that .apk is the "suggested version", then we should eliminate the idea of
"suggested version code" and instead have a "suggested apk" (which implicitly
includes the repository it comes from, so we choose the one with the better
priority). Right now, we kind of assume that it doesn't matter which repo
provides the suggested apk, as long as one of them has an .apk with the correct
version code and signing key.

It shouldn't _particularly_ matter from a security perspective, because
a malicious repo wont be able to trick a user into installing an apk with a
different signing key, but it would be good to iron this out.

This commit adds a TODO explaining this for th ebenefit of any CRer.
2016-10-25 11:54:41 +03:30
Peter Serwylo
470217c050 Finalise tests for repo priorities + app metadata 2016-10-25 11:54:41 +03:30
Peter Serwylo
b3bb602f7d Add test for repo priorities + app metadata. Not passing yet. 2016-10-25 11:54:41 +03:30
Peter Serwylo
04e76a1d5a Moved tests into updater package, updated multiRepo.*.jar repos.
The tests are in the .updater packate to make them easier to run as
a suite in Android Studio. Now the package can be right clicked and
ran to run all the tests to do with updating.

The index jar files were updated so as to include info in most
metadata fields (e.g. URLs/descriptions/summary/etc) to show that
that particular part of metadata came from a specific repo. This
will allow more specific tests to show that we can indeed query for
an app with metadata provided by the repo with the highest priority.
2016-10-25 11:54:41 +03:30
Peter Serwylo
3de53530df Enable multirepo tests, make them pass.
Tests were failing because the repository of existing and new apks were not
considered when deciding whether to update or insert them.
2016-10-25 11:54:41 +03:30
Peter Serwylo
be9690228f WIP: Making metadata table work. Requires significant refactorings throughout :( 2016-10-25 11:54:41 +03:30
Peter Serwylo
05a64084d7 Migrate priorities safely. 2016-10-25 11:54:40 +03:30
Peter Serwylo
b5ba512a71 Added package table.
Required for future work which will be better able to deal with multiple repos
providing the same app.
Instead of migrating data into that table, we will drop and recreate the tables.
This is because before this feature is out, we'll need to do that anyway.
2016-10-25 11:54:40 +03:30
Peter Serwylo
75b93a5457 Fix incorrect version check in db helper. 2016-10-25 11:54:40 +03:30
Peter Serwylo
9170cdc6ac Cleanup DBHelper in prep for package table in the future. 2016-10-25 11:54:40 +03:30
Peter Serwylo
b81d0f9290 Added helper function for debugging SQL queries during development
It is often helpful during debugging to be able to dump the contents
of an SQL result `Cursor` to the debug watch list. This is difficult
to do under normal circumstances. This adds a utility method really
only designed to be used during interactive debugging, which will do
its best to build a `Map` for each row in the `Cursor`. This can then
be used to test queries while the debugger is paused.
2016-10-25 11:54:40 +03:30
Peter Serwylo
19293e8735 When inserting a new repo, assign the priority appropriately.
Even though this is not used yet, it will be a requirement in the
near future for the `RepoProvider` to be the one who decides what
the priority of new repositories is. This will prevent clients of
this provider from specifying wrong priorities that result in gaps
For example, if we accidentally ended up with  priorities of
1, 2, 4, and then 5, this would cause problems if the user tried to
drag the second repo to the position of the 4th repo. It is easier
to do these priority shuffles if we can assume that the priorities
are contiguous.
2016-10-25 11:54:40 +03:30
Peter Serwylo
df0b162512 Renamed generic sounding methods to be more specific.
Originally, I hoped that the arguments a method took would help enough
to differentiate the intent of that method. This was the case for methods
such as `getContentUri()` and `find()`. However they are a little confusing
to work with, so this change renames a bunch of methods to be more specific.
In addition, it makes some renames from app -> package which will help with
the upcoming change to add a `package` table to the database.
2016-10-25 11:54:40 +03:30
F-Droid Translatebot
0362ca7f75 Pull translation updates from Weblate
Translators:

Adrià García-Alzórriz    Catalan
Ajeje Brazorf            Sardinian
Alberto Moshpirit        Spanish
Koen Glotzbach           Dutch
Licaon Kter              Romanian
Marcelo Santana          Portuguese (Brazil)
naofum                   Japanese
Nathan Follens           Dutch
Nutchanon Wetchasit      Thai
Osoitz                   Basque
Sérgio Marques           Portuguese (Portugal)
Sveinn í Felli           Icelandic
Sylvia van Os            Dutch
Verdulo                  Esperanto
Verdulo                  Polish
YFdyh000                 Simplified Chinese
2016-10-25 11:54:40 +03:30
Hans-Christoph Steiner
adcd383404 Installer methods no longer need to accept Apk as an argument
Since e69a6d5a8f24e7745516001f58bee49e05f2ea9e, the Apk instance is
provided in the constructor and is available as a final instance variable.
No need to pass it around.  Thanks to @pserwylo for spotting this.
2016-10-25 11:54:40 +03:30
Hans-Christoph Steiner
d4b697c76e show complete link command in debug logging
Its helpful when debugging if the complete command is printed,
without omitting the last element.
2016-10-25 11:54:40 +03:30
Hans-Christoph Steiner
f0d6acd974 send Installer.ACTION_INSTALL_STARTED consistently
ACTION_INSTALL_STARTED was being sent twice per transaction with the
default installer. Also, it should be sent as the first step of the install
process.
2016-10-25 11:54:40 +03:30
Hans-Christoph Steiner
58959b2e74 add preference to control storing install history
For now, this is disabled by default and hidden in the expert preferences
since it doesn't do anything yet inside of F-Droid.  It is useful now for
whitelabel builds to fetch the install history from another app. #396
2016-10-25 11:54:40 +03:30
Hans-Christoph Steiner
0a1baa9e67 allow install history to be read from an ContentProvider
This allows a designated app to read the install history from F-Droid via a
ContentProvider.  The app is designated by the packageName defined in the
string install_history_reader_packageName.
2016-10-25 11:54:40 +03:30
Hans-Christoph Steiner
c4ff9ecc43 store install/uninstall history for later use
The install and uninstall history has lots of uses, including displaying
to the user in the app itself, reporting to the Device Administrator to
enable tracking of installs/uninstalls from the admin's app repo, etc.  It
can also be used as part of a "popularity contest" #396
2016-10-25 11:54:40 +03:30
Hans-Christoph Steiner
af7840d993 if updating notification, also try getting app name from DB
Now that the packageName is included in the Installer broadcast Intents,
it can be used to fetch the app name from the database, if all other ways
fail.
2016-10-25 11:54:40 +03:30
Hans-Christoph Steiner
897150cba2 store Apk instance in Installer instance for fetching state
If F-Droid or InstallManagerService get killed while an install is in
progress, that install will ultimately broadcast back to
InstallManagerService to manage the notifications.  The state is gone
since things have been killed, so include the Apk instance in the
Intent that is included in the broadcasts so that
InstallManagerService can fetch all required info from the database.

closes #698
2016-10-25 11:54:40 +03:30
Hans-Christoph Steiner
c238c4d4cf prevent crashing if push requests include bad packageNames
F-Droid shouldn't crash if a push request includes a bad package name. This
just makes it silently ignore those push requests.  If its a debug build,
it will send a message to logcat.  I'm not sure this is best way to handle
this, but this is better than crashing the app.  This will make it harder
for repo operators to debug issues with push requests.
2016-10-25 11:54:40 +03:30
Hans-Christoph Steiner
39230c4efb basic support for repo push requests, configed in default_repos
This allows whitelabel versions of apps to specify built-in app repos that
have push requests accepted by default.  This is useful for the case where
there is a central manager of the core apps that are installed.

https://gitlab.com/fdroid/fdroidserver/issues/177
2016-10-25 11:54:40 +03:30
Daniel Martí
370938dda4 gradle: bump version in wrapper to 3.1
Also re-run `gradle wrapper` from 3.1 to update the scripts and jar.
2016-10-25 11:54:40 +03:30
Daniel Martí
3d3d0bb236 gradle: bump plugin version to 2.2.0
Also update the dependency declarations since the current ones break
with the new plugin/gradle version.

https://stackoverflow.com/questions/39602587/could-not-get-unknown-property-assemblerelease-for-project
2016-10-25 11:54:40 +03:30
Hans-Christoph Steiner
a45af269d6 convert default_repo.xml to more flexible format
This is a step towards supporting easy whitelabeling, using gradle flavors.
This allows the whitelabel version to set the default repos just by making
their own default_repos.xml in app/src/whitelabel/res/values.  That one
will then override the built-in F-Droid one.
2016-10-25 11:54:40 +03:30
Hans-Christoph Steiner
58b38a42d4 rename default_repo.xml to default_repos.xml, there are many
Just to make it clear that there can be and are multiple repos configured
in the file.
2016-10-25 11:54:40 +03:30
Daniel Martí
c4f671bb45 Bump versionCode to 0.102-alpha0
This enables upgrading from 0.101 to master until alpha1 is out.
2016-10-25 11:54:40 +03:30
Daniel Martí
c10a886010 Sync changelog with 0.101, add 0.102 2016-10-25 11:54:40 +03:30
Hans-Christoph Steiner
a2e44dcfb0 move CrashReportActivity into new .acra package
This isolates all the ACRA stuff in its own package.
2016-10-25 11:54:40 +03:30
Hans-Christoph Steiner
0a1cab0df8 custom ACRA sender to put stacktrace title in email Subject:
This should hopefully make it easier to sort through the emails.
2016-10-25 11:54:40 +03:30
Hans-Christoph Steiner
74c60bb4c9 throw in extra test URLs for HttpDownloaderTest
These are commented out so that the tests aren't brittle when they run on
the CI builds.  But are easy to uncomment when testing locally.
2016-10-25 11:54:40 +03:30
Hans-Christoph Steiner
ba9f9af76f support TLS 1.2 on all repos
Update to the latest NetCipher, which now fully supports SNI, in order to
support TLS 1.2 on all supported platform levels.  Without this, a repo
that is TLS 1.2 only will be unusable on all but the most recent versions
of Android.

#431
2016-10-25 11:54:40 +03:30
Hans-Christoph Steiner
d281d3f853 move HttpDownloaderTest to emulator, AndroidOS has oddities
There are oddities with the way that Android has implemented the network
stack, as compared to OpenJDK or Oracle JDK.  So running the tests on the
local JVM, i.e. Robolectric, will not provide good test coverage for real
world use cases.
2016-10-25 11:54:40 +03:30
Hans-Christoph Steiner
d8a0563ac0 update changelog for 0.101 2016-10-25 11:54:40 +03:30
Hans-Christoph Steiner
9f08be0c7d update to latest support lib bugfix version: v24.2.1 2016-10-25 11:54:40 +03:30
Hans-Christoph Steiner
4ae8bb3804 gitlab-ci: ensure the SDK is fully updated before running
This makes sure that the latest version of the core SDK components are
all current before running, which means that the builds will keep
working even when the docker image gets out of date. Then we can
finish the task we are working on before having to deal with updating
the docker image. Without that update line, we have to drop everything
and update the docker image when things are out of date.

Disk space and bandwidth is cheap, developer time is very scarce. We
should aim to keep the tests working as much as possible so that we
waste less developer time. Updating everytime only means it downloads
a little bit of XML each time, that's nothing compared to what gradle
downloads on every build.
2016-10-25 11:54:40 +03:30
Hans-Christoph Steiner
6e03fb7a2c gitlab-ci: use before and after script for reused chunks
It also reduces redundancy by using before_script and after_script blocks.
2016-10-25 11:54:40 +03:30
Hans-Christoph Steiner
eeb1ae24a0 gitlab-ci: reduce connected10 from error to warning
connected10 is still flaky since the emulator crashes a lot.  Marking it
for "allow_failure" will still show a warning, but it won't fail the build.
2016-10-25 11:54:40 +03:30
Daniel Martí
a214f8aa06 Add Icelandic to the list of languages
Added to weblate in the last commit.
2016-10-25 11:54:40 +03:30
F-Droid Translatebot
8cbc3c7557 Pull translation updates from Weblate
Translators:

Ajeje Brazorf       Sardinian
Enol P              Asturian
Marian Hanzel       Slovak
Michael Moroni      Italian
Mladen Pejaković    Serbian
Sérgio Marques      Portuguese (Portugal)
Sveinn í Felli      Icelandic
Дмитрий Михирев     Russian
2016-10-25 11:54:40 +03:30
Felix Ableitner
c5fa6d94c7 Fixed link to privileged extension 2016-10-25 11:54:40 +03:30
Peter Serwylo
f61fa0c4f8 More robust fix for #763, specifying column names to copy explicitly.
This is far less brittle at runtime, but slightly more work at dev time.
The following things are undesirable but make it much easier to write:
 * Use of `CREATE_TABLE_APP.replaceFirst(...)` to create the temp tables.
 * Having to specify a list fo columns twice in `Schema` (`ALL_COLS` + `COLS`).

The `replaceFirst` means we don't need to maintain two separate create table
statements. It is a little messy because there is no compile time guarantee
that we are creating a valid SQL statement at the end, just our knowledge
that a create table statment tends to have the table name first and it
probably wont cause problems.

The `ALL_COLS` + `COLS` is required so that we don't have to type out a list
of fields when copying data in `TempAppProvider`. Otherwise, whenever a new
column is added, developers would need to know that it also needs to be added
to this third place. Currently it is in the `Schema` and the `CREATE_TABLE_*`
statements where one needs to add a new column. These are both intuitive and
hopefully easily discoverable. Having to add it to the `TempAppProvider` is
less intuitive and likely to result in bugs.
2016-10-25 11:54:40 +03:30