Fixes two bugs that appeared since the introduction of the struct:
- when reading permissions from the journal, null ("") was read as
empty-not-null
- when reading permissinos from the server, empty ("") was read as null
Addresses #4608
Issue #6420
Store the X-Request-ID in the SyncFileItem and also in the blacklist.
Note that for consistency reason, the X-Request-ID is also in the
SyncFileItem if the request succeeds.
Currently there is no UI to access it, but it can be queried with sql
commands
For conflicts generally as well as new files in read-only directories
the context menu will now present delete and move options.
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
Add an index on the e2eMangledName column in the metadata table
to speed up file sync by orders of magnitude on directories with
a large number of files.
Signed-off-by: Jan Schmidt <jan@centricular.com>
This flagged mostly unused parameters. Didn't enable the
misc-non-private-member-variables-in-classes check as we got a lot of
those. Hopefully we'll get to fix them at some point but that feels too
early and too much work for now.
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
This will allow to exploit the information of a directory being
encrypted or not during the discovery phase.
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
Otherwise on second sync we wouldn't find anything under a mangled path
and just end up thinking said files were removed.
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
Otherwise, after a first sync all the mangled entries would be removed
from the journal. On second sync it would be fine because we'd then have
seen the unmangled names because of the local files. Unfortunately
that'd mean reuploading them for nothing or trying to mkdir again on the
server for nothing... with a chance of using differently mangled names
(although I didn't spot it, I can't exclude it never happened).
This also led to weirdly getting stuck during sync when there was more
than one sync point.
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
Make the codebase consistent, we already have a lot of implicit pointer comparisons.
Exception: Stay explicit on return's, example:
return _db != nullptr;
Signed-off-by: Michael Schuster <michael@schuster.ms>
We keep NULL in the pure C files in src/csync/std and test/csync.
We also replace Doxygen documentation referring to "NULL" to
"\c nullptr" (formatted as code).
Signed-off-by: Stephan Beyer <s-beyer@gmx.net>
This also fixes a couple of warnings at places (out of order init for
instance) and a potential bug in the webflow credentials / qtkeychain
integration.
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
Return value determines which theme is applied for monochrome variant.
Fixes#1276 but only when monochrome option is toggled or nc starts, not automatically when changing windows theme.
Signed-off-by: Dominique Fuchs <32204802+DominiqueFuchs@users.noreply.github.com>
Since commit 4dc49ff3, we store an entry in the upload info table even
for non chunked uploads. However, if this fails we don't want to remove
non-existant stale chunks if the upload fails.
Without this commit, we would send a DELETE command to clean non-existant
chunks in the dav/uploads/ namespace.
Before, we only cleared the _avoidReadFromDbOnNextSyncFilter *after* a
sync which meant that we had to sync twice after selective sync setup.
Now, we clear the filter *before* a sync as well which allows the actual
next sync to write the correct etags to the db again - instead of only
the sync after that one.
Also expand on comments and rename _avoidReadFromDbOnNextSyncFilter to
_etagStorageFilter.
Since the release package will be build with unit test, we don't
want to query the env variable at every call to fsCasePreserving.
So only test the env variable at startup.
And the testutility can still change the value.
(The env variable is still used from t8.pl and maybe smashbox)
Issue #6318
Paths can contain the wildcards % and _ and that would lead to odd
behavior.
This patch also clarifies the behavior of avoidReadFromDbOnNextSync()
which previously dependend on whether "foo/bar" or "foo/bar/" was
passed as input.
Possibly affects #6322
(PR #6265)
- Remove the UI completely
- Move the #ifdef inside the FileSystem::moveToTrash function, so it is easier to
implement on other platforms
- Q_OS_UNIX includes mac, so we need to disable it. (not using Q_OS_LINUX because of
other BSD that uses XDG spec as well
- Translate the error messages
- Add a couple of doc comments
QFileInfo::isSymLink() does detect reparse points that are symlinks but
returns false for junctions. The new function FileSystem::isJunction()
can detect those and is used to not recursively delete files inside
directories that are junctions.
See also https://bugreports.qt.io/browse/QTBUG-45344 and the
discussion in the PR https://codereview.qt-project.org/#/c/113019/.
When populating the tree from the filesystem we need to fill
the e2eMangledName from DB and we when trying to find another
match on the oposite tree we need to take in account that names
don't match and search with the mangled name information
If the code was not complex enough syncing two tables
already started to give UNIQUE constrains errors on
simple sync operations, this also adds initial support
remote delete of an encrypted file
If the server has the 'uploadConflictFiles' capability conflict
files will be uploaded instead of ignored.
Uploaded conflict files have the following headers set during upload
OC-Conflict: 1
OC-ConflictBaseFileId: 172489174instanceid
OC-ConflictBaseMtime: 1235789213
OC-ConflictBaseEtag: myetag
when the data is available. Downloads accept the same headers in return
when downloading a conflict file.
In the absence of server support clients will identify conflict files
through the file name pattern and attempt to deduce the base fileid.
Base etag and mtime can't be deduced though.
The upload job for a new conflict file will be triggered directly from
the job that created the conflict file now. No second sync run is
necessary anymore.
This commit does not yet introduce a 'username' like identifier that
automatically gets added to conflict file filenames (to name the files
foo_conflict-Fred-1345.txt instead of just foo_conflict-1345.txt).
Previously, there was csync_ftw_type_e and SyncFileItem::Type. Having
two enums lead to a bug where Type::Unknown == Type::File that went
unnoticed for a good while.
This patch keeps only a single enum.
This can happen if the upload of a file is finished, but we just got
disconnected right before recieving the reply containing the etag.
So nothing was save din the DB, and we are not sure if the server
recieved the file properly or not. Further local update of the file
will cause a conflict.
In order to fix this, store the checksum of the uploading file in
the uploadinfo table of the local db (even if there is no chunking
involved). And when we have a conflict, check that it is not because
of this situation by checking the entry in the uploadinfo table.
Issue #5106
In addition to using the right function when retrieving inodes this
*also* fixes a more general bug ownsql had with storing uint64 values
that didn't fit into an int64.
The assert was made fatal when looking at asserts for #5429#5518
without having a particular problem in mind. Recent reports weakly
suggest that this might lead to occasional crashes for users when
sqlite_close fails in ways that look ignorable.
When users share the same tree several times (say A/ and A/B/ are both
shared) the remote tree can have several entries that have the same
file id. This needs to be respected in rename detection.
Also adds several tests and fixes for issues noticed during testing.
See #6096
We mostly trust the file watchers meaning that we don't re-scan the
local tree if we have done that recently and no file watcher events
have arrived. If the file watchers invalidate a subtree, we rescan
only that subtree.
Since we're not entirely sure the file watchers are reliable, we still
do full local discoveries regularly (1h by default). There is a config
file setting as well as an environment variable to control the interval.
This gets rid of the csync_statedb sqlite layer and use
the same code and same connection as the rest of the SyncEngine.
Missing functions are added to SyncJournalDb and change a few minor
things (like changing SyncJournalFileRecord::_modtime to be an int64
instead of a QDateTime, like it was in csync).
The current implementation would return the same value whether the query failed
or if no row would be found. This is something that is currently checked by csync
and needs to be provided if we want to use SyncJournalDB there.
Adjusted all call sites to also check the return value even though they
could still just rely on rec.isValid(), but makes it more explicit as to what
happens for database errors in those cases, if we ever want to gracefully handle
them.
Create a specific type that parses the permissions so we can store
it in a short rather than in a QByteArray
Note: in RemotePermissions::toString, we make sure the string is not
empty by adding a space, this was already existing before commit
e8f7adc7ca where it was removed by mistake.
This is motivated by the fact that QMetaObject::noralizeSignature takes 7.35%
CPU of the LargeSyncBench. (Mostly from ABstractNetworkJob::setupConnections and
PropagateUploadFileV1::startNextChunk). It could be fixed by using normalized
signature in the connection statement, but i tought it was a good oportunity
to modernize the code.
This commit only contains calls that were automatically converted with clazy.
Set OWNCLOUD_UPLOAD_CONFLICT_FILES=1 to trigger this behavior.
Note that this is experimental and unsupported. The real feature is
likely to end up in 2.5.
Uploading conflict files is simply done by removing the pattern from
csync_exclude. The rest here deals with making the conflict notification
ui approximately work.
There are still some concerns about where an uploaded conflict file
appears in the sync protocol and issues list (it should be in both, but
is only in one of them currently!).
See #4557.
* The sharing ui does a propfind anyway: use that to query the new
property as well!
* For the socket api, asynchronously query the server for the right url
when an action that needs it is triggered.
The old, manually generated URL will be used as fallback in case the
server doesn't support the new property or the property can't be
retrieved for some reason.
Depends on owncloud/core#29021
When created from scratch, the owncloud icon won't be picked-up by
explorer unless the right attributes are set on both the sync root and the
Desktop.ini file.
Issue #2446
Also move csync_normalize_etag to common/utility since we
don't need the char* function anymore.
Remove the single space file_stat->remotePerm codepath since
this won't be used in csync anymore since
8de3bda0b1.
Issue #1817
Now that csync builds as C++, this will avoid having to implement
functionalities needed by csync mandatorily in csync itself.
This library is built as part of libocsync and symbols exported
through it.
This requires a relicense of Utility as LGPL. All classes moved into
this library from src/libsync will need to be relicensed as well.
Christian Kamm