1
0
Fork 0
Commit Graph

253 Commits

Author SHA1 Message Date
Tomaz Canabrava d24a1e542e Merge branch 'master' into clientSideEncryptionV3 2018-01-29 14:06:12 +01:00
Tomaz Canabrava 7b123650da [CSE] Search for name, not mangled name. 2018-01-28 23:48:25 +01:00
Tomaz Canabrava 0c94142c97 [CSE] More debug output 2018-01-28 23:47:58 +01:00
Tomaz Canabrava a003d216a5 [CSE] Implementation of the setE2eRelation
This call creates a new entry on the database for e2e.
2018-01-28 18:05:28 +01:00
Tomaz Canabrava 32eaaa0601 [CSE] GetMangledName implementation 2018-01-28 17:50:10 +01:00
Tomaz Canabrava be817748f8 [CSE] Reset the new Queries on close 2018-01-28 17:46:20 +01:00
Tomaz Canabrava b909bb6977 [CSE] Change the Database to handle Encrypted Files
Create a table to hold the information between the
real file name and the fake file name
2018-01-28 16:41:07 +01:00
Olivier Goffart 776bbbf7b1 Don't include sqlite3.h from headers
So that sqlite is not part of the public interface of csync
(and that the sqlite include path don't need to be passed when compiling
libsync or gui)
2018-01-13 13:58:17 +01:00
Dominik Schmidt dbf15fb8dc Make setLaunchOnStartup debug output more meaningful 2018-01-11 16:11:23 +01:00
Christian Kamm 3988f3ad77 GCC: Fix implicit-fallthrough warnings 2018-01-05 09:43:53 +01:00
Daniel Nicoletti a63d34f870 Prepend "nextcloud" for all logging categories
Thus making easier to exclude logging from kio, qt
and only enable "nextcloud.*"
2017-12-28 17:33:10 -02:00
Christian Kamm a7c0cfc8eb Upload conflict files #4557
If the server has the 'uploadConflictFiles' capability conflict
files will be uploaded instead of ignored.

Uploaded conflict files have the following headers set during upload
  OC-Conflict: 1
  OC-ConflictBaseFileId: 172489174instanceid
  OC-ConflictBaseMtime: 1235789213
  OC-ConflictBaseEtag: myetag
when the data is available. Downloads accept the same headers in return
when downloading a conflict file.

In the absence of server support clients will identify conflict files
through the file name pattern and attempt to deduce the base fileid.
Base etag and mtime can't be deduced though.

The upload job for a new conflict file will be triggered directly from
the job that created the conflict file now. No second sync run is
necessary anymore.

This commit does not yet introduce a 'username' like identifier that
automatically gets added to conflict file filenames (to name the files
foo_conflict-Fred-1345.txt instead of just foo_conflict-1345.txt).
2017-12-15 18:03:53 +01:00
Christian Kamm 8eebc53728 Unify item type enum
Previously, there was csync_ftw_type_e and SyncFileItem::Type. Having
two enums lead to a bug where Type::Unknown == Type::File that went
unnoticed for a good while.

This patch keeps only a single enum.
2017-12-15 18:02:04 +01:00
Olivier Goffart 4dc49ff3b0 SyncEngine: Recover when the PUT reply (or chunkin's MOVE) is lost
This can happen if the upload of a file is finished, but we just got
disconnected right before recieving the reply containing the etag.
So nothing was save din the DB, and we are not sure if the server
recieved the file properly or not. Further local update of the file
will cause a conflict.

In order to fix this, store the checksum of the uploading file in
the uploadinfo table of the local db (even if there is no chunking
involved).  And when we have a conflict, check that it is not because
of this situation by checking the entry in the uploadinfo table.

Issue #5106
2017-12-14 11:56:12 +01:00
Christian Kamm e5ed8fc90a Merge remote-tracking branch 'origin/2.4' 2017-12-13 11:03:24 +01:00
Christian Kamm dcf0baa9de Journal: 64bit inodes, fix storing uint64s generally
In addition to using the right function when retrieving inodes this
*also* fixes a more general bug ownsql had with storing uint64 values
that didn't fit into an int64.
2017-12-12 10:30:54 +01:00
Olivier Goffart 1a503d54f6 Remove usage of deprecated qSort 2017-12-08 16:15:47 +01:00
Olivier Goffart 263668b1cb Remove use of deprecated translate call 2017-12-08 16:15:38 +01:00
Olivier Goffart 3df65460f5 Remove usage of QString::null
Replaces by "QString()"
2017-12-08 16:15:22 +01:00
Olivier Goffart 74672d493d Utility: use QUrlQuery
For QUrl::setQuery is deprecated in Qt5
2017-12-08 16:15:17 +01:00
Olivier Goffart ed5385faa3 Merge remote-tracking branch 'origin/2.4'
Conflicts:
	src/libsync/connectionvalidator.cpp
     	src/libsync/networkjobs.cpp
2017-12-05 15:58:13 +01:00
Markus Goetz 6276d9290d utility_mac: Check for NULL from LSSharedFileListItemResolve #6123
Possibly also #6074
2017-11-28 13:35:15 +01:00
Christian Kamm aafe7c4732 Merge remote-tracking branch 'origin/2.4' 2017-11-20 11:54:56 +01:00
Olivier Goffart 480932a58a Checksum: Ignore unkown OC-Checksum header when downloading...
And if there are several checksums, pick the "best" one.

The case of several checksum was reported in
https://github.com/nextcloud/client_theming/issues/213
2017-11-20 10:51:15 +01:00
Christian Kamm 820ebf4b6e Sql: db close() failing isn't fatal
The assert was made fatal when looking at asserts for #5429 #5518
without having a particular problem in mind. Recent reports weakly
suggest that this might lead to occasional crashes for users when
sqlite_close fails in ways that look ignorable.
2017-11-14 12:10:19 +01:00
Christian Kamm 3608114ec1 Journal: Fix low-disk space errors
Fixes a regression from 4dbe9d4113,
low disk space can lead to IOERR not just CANTOPEN.
2017-11-03 10:44:47 +01:00
Christian Kamm 4dbe9d4113 Journal: Don't crash if the db file is readonly #6050
Surprisingly sqlite3_open_v2() returns ok on readonly files with
SQLITE_OPEN_READWRITE. Probably due to the journal mode?
2017-11-02 16:36:51 +01:00
Christian Kamm 05c1bfb6cf Merge remote-tracking branch 'origin/2.4' 2017-10-27 10:29:31 +02:00
Jocelyn Turcotte 56e38e6f80 Windows: Add sync folders to Explorer's navigation pane
This is only the navigation pane, the SyncRootManager entries aren't handled yet.

This follows the instructions from:
https://msdn.microsoft.com/en-us/library/windows/desktop/dn889934%28v=vs.85%29.aspx

Issue #5295
2017-10-24 16:10:53 +02:00
Christian Kamm 9c7ee6ef85 Reconcile: Rename handling fixes: duplicate file ids
When users share the same tree several times (say A/ and A/B/ are both
shared) the remote tree can have several entries that have the same
file id. This needs to be respected in rename detection.

Also adds several tests and fixes for issues noticed during testing.

See #6096
2017-10-24 10:54:23 +02:00
Christian Kamm e85a339d94 Local discovery: Use db instead of filesystem
We mostly trust the file watchers meaning that we don't re-scan the
local tree if we have done that recently and no file watcher events
have arrived. If the file watchers invalidate a subtree, we rescan
only that subtree.

Since we're not entirely sure the file watchers are reliable, we still
do full local discoveries regularly (1h by default). There is a config
file setting as well as an environment variable to control the interval.
2017-10-24 10:00:20 +02:00
Olivier Goffart a9761a8976 Use qEnvironmentVariable* instead of qgetenv when appropriate
Now that we use Qt5, we should do that. It is slightly more efficient
and declares the intent.
(Modified using clazy)
2017-10-19 13:57:49 +02:00
Markus Goetz d339b68715 Checksums: Use addData function
Our implementation had bad error handling.
This one now uses a new Qt5 addData function.

owncloud/enterprise#2252
2017-10-17 13:26:54 +02:00
Christian Kamm 3f7b3ca962 Checksums: Improve logging and add global disable #5017 2017-10-13 13:08:20 +02:00
Jürgen Weigert 28fe903aee support old sqlite before 3.7.7
quickfix: define the symbol for backwards compat
or FIXME: better solution #ifdef SQLITE_IOERR_SHMMAP where needed...
2017-09-28 19:01:55 +02:00
Olivier Goffart 13e0cc6b50 Use QDateTime::currentDateTimeUtc instead of the non utc version
QDateTime::currentDateTime is terribly slow.
2017-09-26 16:14:17 +02:00
Olivier Goffart 278129b286 OwnSQL: don't parse the error message when we are done 2017-09-26 11:03:23 +02:00
Jocelyn Turcotte 54c522c861 SyncJournalDb: Reimplement the db_is_empty logic from csync
This reduces the initial sync local discovery time from
2.0 to 0.6 seconds in LargeSyncBench on my machine.
2017-09-26 11:03:23 +02:00
Jocelyn Turcotte a034ee894c Use SyncJournalDb in csync
This gets rid of the csync_statedb sqlite layer and use
the same code and same connection as the rest of the SyncEngine.

Missing functions are added to SyncJournalDb and change a few minor
things (like changing SyncJournalFileRecord::_modtime to be an int64
instead of a QDateTime, like it was in csync).
2017-09-26 11:03:23 +02:00
Jocelyn Turcotte 6f46764daa SyncJournalDB: Allow callers of getFileRecord if the query failed
The current implementation would return the same value whether the query failed
or if no row would be found. This is something that is currently checked by csync
and needs to be provided if we want to use SyncJournalDB there.

Adjusted all call sites to also check the return value even though they
could still just rely on rec.isValid(), but makes it more explicit as to what
happens for database errors in those cases, if we ever want to gracefully handle
them.
2017-09-26 11:03:23 +02:00
Jocelyn Turcotte e1d482ea7c SyncJournalDB: Bind phash as a qlonglong instead of a QString
This is what csync does and we don't need to pass it as a string
to let sqlite convert it back to a uint64 later.
2017-09-26 11:03:23 +02:00
Olivier Goffart 95d23b1914 RemotePermissions: Store in a class rather than in a QByteArray to save memory
Create a specific type that parses the permissions so we can store
it in a short rather than in a QByteArray

Note: in RemotePermissions::toString, we make sure the string is not
empty by adding a space, this was already existing before commit
e8f7adc7ca where it was removed by mistake.
2017-09-22 09:29:08 +02:00
Olivier Goffart ff4213b59f Use the Qt5 connection syntax (automated with clazy)
This is motivated by the fact that QMetaObject::noralizeSignature takes 7.35%
CPU of the LargeSyncBench. (Mostly from ABstractNetworkJob::setupConnections and
PropagateUploadFileV1::startNextChunk). It could be fixed by using normalized
signature in the connection statement, but i tought it was a good oportunity
to modernize the code.

This commit only contains calls that were automatically converted with clazy.
2017-09-21 14:05:39 +02:00
Christian Kamm 30843cd368 Fix compile with bad strrchr signature 2017-09-19 12:44:59 +02:00
Christian Kamm 53195b064a Conflicts: Upload them files if env variable says so
Set OWNCLOUD_UPLOAD_CONFLICT_FILES=1 to trigger this behavior.

Note that this is experimental and unsupported. The real feature is
likely to end up in 2.5.

Uploading conflict files is simply done by removing the pattern from
csync_exclude. The rest here deals with making the conflict notification
ui approximately work.

There are still some concerns about where an uploaded conflict file
appears in the sync protocol and issues list (it should be in both, but
is only in one of them currently!).

See #4557.
2017-09-19 11:45:38 +02:00
Christian Kamm dca2664707 Private links: Retrieve link through propfind property #6020
* The sharing ui does a propfind anyway: use that to query the new
property as well!
* For the socket api, asynchronously query the server for the right url
when an action that needs it is triggered.

The old, manually generated URL will be used as fallback in case the
server doesn't support the new property or the property can't be
retrieved for some reason.

Depends on owncloud/core#29021
2017-09-19 10:51:03 +02:00
Jocelyn Turcotte 5e332983e2 SyncJournalDB: Only reset_and_clear_bindings once
We only need to do it once between each query execution and
that can be right before the first bind.
2017-09-18 14:00:52 +02:00
Jocelyn Turcotte a1f1775d15 Move SyncJournalDB to src/common 2017-09-18 14:00:52 +02:00
Jocelyn Turcotte bd25225045 Move non-csync-dependent parts of FileSystem to src/common
This keep the csync-dependent parts in src/libsync, slitting the namespace
over two files.
This will allow moving SyncJournalDB to src/common as well.
2017-09-18 14:00:52 +02:00
Helmut K. C. Tessarek 709aa27031 remove qt4 code 2017-09-15 07:11:05 +02:00
Jocelyn Turcotte 2d7a4bdb06 Windows: Make sure that the folder and Desktop.ini have the right attributes (#6018)
When created from scratch, the owncloud icon won't be picked-up by
explorer unless the right attributes are set on both the sync root and the
Desktop.ini file.

Issue #2446
2017-09-11 15:32:42 +02:00
Jocelyn Turcotte e8f7adc7ca Replace csync_vio_file_stat_t with csync_file_stat_t
Also move csync_normalize_etag to common/utility since we
don't need the char* function anymore.

Remove the single space file_stat->remotePerm codepath since
this won't be used in csync anymore since
8de3bda0b1.

Issue #1817
2017-09-06 13:08:23 +02:00
Jocelyn Turcotte cf15cbf0b3 Move Utility to a new common static library
Now that csync builds as C++, this will avoid having to implement
functionalities needed by csync mandatorily in csync itself.

This library is built as part of libocsync and symbols exported
through it.
This requires a relicense of Utility as LGPL. All classes moved into
this library from src/libsync will need to be relicensed as well.
2017-09-05 17:25:19 +02:00