1
0
mirror of https://github.com/chylex/Nextcloud-Desktop.git synced 2024-09-28 06:42:46 +02:00
Commit Graph

7999 Commits

Author SHA1 Message Date
Hannah von Reth
4681421b62
Remvoe now unneeded SyncRunFileLog::instructionToStr 2020-12-15 10:59:11 +01:00
Hannah von Reth
a1d8010eae
Cleanup moc_csync.cpp include 2020-12-15 10:59:11 +01:00
Hannah von Reth
4d615c31da
Rename csync_instructions_e -> SyncInsturctions 2020-12-15 10:59:10 +01:00
Hannah von Reth
8cbdb4451b
Remove now unneede csync_instruction_str 2020-12-15 10:59:10 +01:00
Hannah von Reth
caa04f6adb
Make bit flags better readable 2020-12-15 10:59:10 +01:00
Hannah von Reth
c803292829
Use Q_ENUM_NS for PinState 2020-12-15 10:59:10 +01:00
Hannah von Reth
38c4d5a406
Cookies: Do set cookies in DetermineAuthTypeJob too
As we don't support cookie based authentication anymore we can provide cookies here.
This fixes issues with loadbalancers access policy managers.
2020-12-15 10:59:10 +01:00
Hannah von Reth
f073997aee
Cookies: Don't override cookies with outdated values
This code was actually not breaking most cookie handling by accident.
As the raw cookies where not split properly we added cookies with values like
"key: val; key2 = val2; key3 = val3"
When the code was corrected we overwrote the newer values in the jar with
the old ones from a request.
2020-12-15 10:59:09 +01:00
Olivier Goffart
317d3735d7
SocketAPI: fix status after a failure to move
Issue #7759
2020-12-15 10:59:09 +01:00
Hannah von Reth
85aefa4232
Sync: Fix handling of virtual files in error state
Issue: #7799
2020-12-15 10:59:09 +01:00
Hannah von Reth
bcceb5c33d
[Gui] Implement raiseDialog on Windows
Issue: #7774
2020-12-15 10:59:09 +01:00
Hannah von Reth
12ae1e97c6
[SocketApi] Ensure listener still exists 2020-12-15 10:59:08 +01:00
Hannah von Reth
868b05f25b
Improve logging of issues during plugin loading
If the plugin could not be loaded the client calls qFatal
Make the loading warnings critical so they get printed before we crash
2020-12-15 10:59:08 +01:00
Hannah von Reth
d63d4cdf62
Don't insert items into the folder model if selectiveSync is not supported
This fixes an assertion in FolderStatusModel::SubFolderInfo::resetSubs
rowCount reported 0 but we actually had items in the model
2020-12-15 10:59:08 +01:00
Hannah von Reth
9176afe263
Make sure all cases are handled 2020-12-15 10:59:08 +01:00
Hannah von Reth
9b0788bdf8
[Gui] Fix vfs dialog not showing on Windows 2020-12-15 10:59:08 +01:00
Hannah von Reth
a9728b527a
Remove unused defines 2020-12-15 10:59:08 +01:00
Hannah von Reth
1dd01477c5
Use Q_ENUM_NS to pretty print csync enums 2020-12-15 10:59:07 +01:00
Hannah von Reth
5927beb08d
Fix potential null pointer access
https://sentry.io/organizations/owncloud/issues/1529161263/events/02509984b5ca42ffb3960d9c9e161414/?project=79001&statsPeriod=14d
2020-12-15 10:59:07 +01:00
Olivier Goffart
b7193e6a0e
Folder Wizard: warn when adding a folder which is the parent of a sync'ed folder
issue #7741
2020-12-15 10:59:07 +01:00
Hannah von Reth
13c9d6431d
Fix expansion of tree view on newly added accounts
The change is based on 97ce20ac028660e6ae3dd0b98d4b487999d8768a

I removed a few lines of code there which are already part of fetchMore()

Fixes: #7336
2020-12-15 10:59:07 +01:00
Hannah von Reth
fdc3b7c8da
[Wizard] Make vfs dialog blocking
Calling the callback after the receiver was deleted caused a crash
Fixes: #7709
Fixes: #7711
2020-12-15 10:59:07 +01:00
Olivier Goffart
a6a0e361c1
fixup test 2020-12-15 10:59:07 +01:00
Hannah von Reth
106a35d242
[SSL] Print warning only if something changed 2020-12-15 10:59:06 +01:00
Hannah von Reth
ba87fc9e78
[SSL] Properly restore user accepted certificats 2020-12-15 10:59:06 +01:00
Hannah von Reth
c8dd333e31
[Sanity] Remove old ifdef 2020-12-15 10:59:06 +01:00
Hannah von Reth
9564e5e92e
Fix saving of cookies
Fixes: #7700
2020-12-15 10:59:06 +01:00
Hannah von Reth
3a3ccb0834
Don't hardcode PLUGINDIR 2020-12-15 10:59:06 +01:00
Olivier Goffart
7fd4a280f0
Make PluginFactory virtual to silent -Wnon-virtual-dtor warning
This is not necessary, but it also shouldn't hurt.
2020-12-15 10:59:06 +01:00
Olivier Goffart
1c10fceacc
SyncEngine: no need to use QAtomicInt
This was done because the propagator jobs where running in a thread a long
time ago, but this is no longer the case.

(Also QAtomicInt::load is marked as deprecated now)
2020-12-15 10:59:05 +01:00
Olivier Goffart
f81f96915f
ProxyAuthHandler Use QPointer instead of QWeakPointer
QWeakPointer::data is deprecated.
In this case we should use QPointer
2020-12-15 10:59:05 +01:00
Hannah von Reth
466e8abc91
[Core] Include more information about the OS in the user agent 2020-12-15 10:59:05 +01:00
Hannah von Reth
edb51abdfd
Don't ignore file sync notification after an unlock
For a usual file sync event we check for actual changes in the local file,
after an unlock the local file might be unchanged so we need to sync it anyhow.

Fixes: owncloud/enterprise#3609
2020-12-15 10:59:05 +01:00
Hannah von Reth
3317e354f2
Prepend PLUGINDIR else its pointless 2020-12-15 10:59:05 +01:00
Hannah von Reth
b6e8d47644
Use Q_ENUM on AuthType to get the name printed in the log 2020-12-15 10:59:04 +01:00
Hannah von Reth
dabf7aaebc
Application: Allow to quit running instances by commandline 2020-12-15 10:59:04 +01:00
Hannah von Reth
fa82a4aff3
Cleanup 2020-12-15 10:59:04 +01:00
Hannah von Reth
6c19b02888
Disable http2 support for now
Issue: #7610
2020-12-15 10:59:04 +01:00
Olivier Goffart
0e7c56e81c
Don't show the "All files deleted" popup when unselecting everything with selective sync
Issue #7337
2020-12-15 10:59:04 +01:00
Olivier Goffart
4424eb7f07
AccountSettings: limit the clickable region of the 'add folder' button
Issue #7326
2020-12-15 10:59:03 +01:00
Olivier Goffart
b97c0ed8a2
Network Settings: Show a warning that proxy settings do not apply to localhost
Only show this if at least one account is detected to have an url that looks
like localhost, because this could otherwise be confusing

Issue #7169
2020-12-15 10:59:03 +01:00
Olivier Goffart
392d3c257c
Discovery: Allow more HTTP error code to be treated as ignored dir
The original code from csync was stopping at any error.
But we have been whitelisting soeme http error code one by one
to ignore the directory instead of aborting the sync.
However, as there are more requests to continue the sync in case
of error, just ignore most HTTP errors

Issue #7586
2020-12-15 10:59:03 +01:00
Hannah von Reth
43c7e32ee1
[libsync] Export verifyFileUnchanged
This fixes Windows builds that are broken since #7562 was merged
2020-12-15 10:59:03 +01:00
Olivier Goffart
66f7b27121
VFS: Do not overwrite existing files by placeholder
For issue #7557 and #7556

Note: this change the API of the VFS plugin, so the VFS plugin needs small
adaptations
2020-12-15 10:59:03 +01:00
Hannah von Reth
9807285abd
[Gui] Set proper hdpi attributes 2020-12-15 10:59:03 +01:00
Olivier Goffart
fc0aeb53b1
Discovery: fix TestSyncVirtualFiles::testExtraFilesLocalDehydrated on windows
On windows we do a test to know if we should change the case of the files,
but that conflict with the test that checks if the file was still there
when the filename is actually the same. Which can happen with virtual files
as they have two representation (the one with and without suffix).
2020-12-15 10:59:02 +01:00
Christian Kamm
4c4cbf0d97
Vfs: Lots of tests and corrections for suffix edge cases
Avoid or deal with problems that happen when suffixed files exist on the
server or suffix and non-suffixed files exist locally.

See #7350, #7261.
2020-12-15 10:59:02 +01:00
Olivier Goffart
88f86a56b1
Discovery: Attempt to fix issue with windows VFS and new files
(or moved files)
As seen in the log of #7558, a conflict may be issued by mistake.

See investigation in https://github.com/owncloud/client/issues/7558#issuecomment-547385362

This hopefully fix #7558
2020-12-15 10:59:02 +01:00
Olivier Goffart
513b0c723c
Account Settings: change the color of info message from green to blue
To avoid confusion with the color of "success"

Issue #7403
2020-12-15 10:59:02 +01:00
Olivier Goffart
4d7ed8f62c
Wizard: fix compilation warnings about unused variables 2020-12-15 10:59:01 +01:00
Olivier Goffart
115a53134c
SyncEngine: Save a bit of memory by not keeping a set of all filename
This is only used for conflict files, so only save conflict files.
(The _seenFile was used for other things in 2.5, but not anymore)
2020-12-15 10:59:01 +01:00
Olivier Goffart
83d743b66b
When moving is allowed but deleting is not, do not restore moved items
Issue #7293
2020-12-15 10:59:01 +01:00
Olivier Goffart
ee611e6004
Restoration items should appear in the sync protocol
When an item is downloaded because it is restored, it shall be shown in the
sync protocol.
(It is also going to be shown in the not synchronized for a short while, but
that's fine)
2020-12-15 10:59:01 +01:00
Christian Kamm
9d422284af
Discovery: Distinguish readdir and closedir errors 2020-12-15 10:59:01 +01:00
Olivier Goffart
adbd3d869b
Fix warning about serverJob not being used
And fix a FIXME in the same time
2020-12-15 10:59:00 +01:00
Christian Kamm
89216daee6
Propagate dir: Never write the etag on remote mkdir #7481
It must always only be written once all children are successfully
propagated.
2020-12-15 10:59:00 +01:00
Hannah von Reth
dc55617144
Ensure the url is complete before we validate it
Fixes: #6722
2020-12-15 10:59:00 +01:00
Christian Kamm
c9476a11f4
Checksums: Explicitly close file before reporting result
To ensure it's no longer open when the finished signal fires.
2020-12-15 10:59:00 +01:00
Christian Kamm
e91e1ca78f
Test: Disable local discovery parallelism in permission test
Adding parallelism broke the test because it depended on the order of
discovery.
2020-12-15 10:59:00 +01:00
Christian Kamm
e596b55977
Discovery: Change local job signal signatures
To make it more explicit that data a copy of the data is transfered
between threads. (though Qt will make a copy of the arguments anyway)
2020-12-15 10:58:59 +01:00
Christian Kamm
71f71b38f1
Discovery: local job shouldn't be parented
Since it'll be deleted by the thread pool.
2020-12-15 10:58:59 +01:00
Christian Kamm
53a217d4e4
Don't store pointer to local job
There were crashes in the QPointer assignment. Possibly the thread pool
is done with the job and deletes it before the assignment is done.
2020-12-15 10:58:59 +01:00
Markus Goetz
26b5e36351
Discovery: List local directories from thread #7456 #7439 2020-12-15 10:58:59 +01:00
Markus Goetz
09a0dbbf82
Reconcile: Sort already during discovery #7445 2020-12-15 10:58:59 +01:00
Christian Kamm
a5c6612883
ConnectionValidator: increase timeout to 57s #7456
When the gui thread blocks for several seconds it's possible for the
ConnectionValidator to timeout and decide that the account is
unreachable. It will then terminate all sync runs.

Increasing the timeout makes this less likely to happen. The tradeoff is
that real disconnects will not be detected as quickly.

This does not address the root cause but makes the symptom less likely
to appear.
2020-12-15 10:58:59 +01:00
Markus Goetz
41dc68f99c
Propagator: Delay job execution a bit #7439 2020-12-15 10:58:58 +01:00
Christian Kamm
e37e954720
Discovery: Remove level of indent around moves
No code changes.
2020-12-15 10:58:58 +01:00
Christian Kamm
28797baa39
Discovery: If a move is forbidden, restore the source
Previously the source was deleted (or attempted to be deleted), even if
the new location was not acceptable for upload. This could make data
unavilable on the server.

For #7410
2020-12-15 10:58:58 +01:00
Christian Kamm
c9d1037622
Propagation: Fix delete-before-rename bug #7441
By introducing a PropagateRootDirectory job that explicitly
separates the directory deletion jobs from all the other jobs.

Note that this means that if there are errors in subJobs the
dirDeletionJobs won't get executed.
2020-12-15 10:58:58 +01:00
Markus Goetz
475117dd60
Propagator: Make sure we schedule only one job #7439
To not starve the event loop.
(There is still ~= 3 jobs running at the same time)
2020-12-15 10:58:58 +01:00
Christian Kamm
3446412d92
PropagateDownload: Don't try to open readonly temporaries
This situation could arrise when receiving a read-only share and the
temporary rename failed for some reason.

See #7419
2020-12-15 10:58:58 +01:00
Hannah von Reth
f78c4f851b
Change Windows virtual files description to tech preview 2020-12-15 10:58:57 +01:00
Christian Kamm
0ac8a3e6be
Don't fatal on "Storage temporarily unavailable"
This is an unreliable workaround. The real fix will need to be deferred
to another release.

For #5088
2020-12-15 10:58:57 +01:00
Hannah von Reth
afc9cd2f46
OwncloudWizard: Mark vfs as tech preview instead of experimental 2020-12-15 10:58:57 +01:00
Hannah von Reth
e5a36c3bc6
CMake VFS: Enable us to provided vfs plugins from an external directory 2020-12-15 10:58:57 +01:00
Dominik Schmidt
7e4d24de2b
SocketApi: Avoid unneccessary conversions 2020-12-15 10:58:57 +01:00
Dominik Schmidt
06ac1c33e8
SocketApi: cleanup debug output 2020-12-15 10:58:57 +01:00
Dominik Schmidt
25cd52dd9f
Add ASYNC_ASSERT_ICON_IS_EQUAL command to SocketApi 2020-12-15 10:58:57 +01:00
Dominik Schmidt
d1f9b1a4f8
Make findWidget more powerful 2020-12-15 10:58:56 +01:00
Dominik Schmidt
67656fb136
Add Q_PROPERTYs for gui testing 2020-12-15 10:58:56 +01:00
Dominik Schmidt
596154a01a
Switch GUI Testing SocketApi to Utf8 2020-12-15 10:58:56 +01:00
Dominik Schmidt
3288a36da6
Add GUI testing SocketApi extension 2020-12-15 10:58:56 +01:00
Hannah von Reth
e97784bb9d
Don't use mutable lambdas as they hide where we modify work on a copy 2020-12-15 10:58:55 +01:00
Hannah von Reth
0f92713ce5
AccountSettings: Use switch, case to ensure we handle all cases 2020-12-15 10:58:55 +01:00
Christian Kamm
bade7aedc6
Checksums: Add back QIODevice api
Because the winvfs plugin needs it. But be more careful about the
device's lifetime this time.
2020-12-15 10:58:55 +01:00
Christian Kamm
18e1098e38
Remove maxLogLines config option
It's no longer used.

For owncloud/docs#1365
2020-12-15 10:58:55 +01:00
Christian Kamm
ea829f96ca
Download: Don't trigger too many concurrent hash computations
Previously the job would only become "active" when the downloads
started. That meant that arbitrarily many hash computations could be
queued at the same time.

Since the the file was opened during future creation this could lead to
a "too many open files" problem if there were lots of new-new conflicts.

To change this:
- Make PropagateDownload become active when computing a hash
  asynchronously.
- Make the computation future open the file only once it gets run. This
  will make it less likely for this problem to occur even if thousands
  of these futures are queued.

For #7372
2020-12-15 10:58:55 +01:00
Christian Kamm
8a5a185752
Fix 'unused parameter name' warnings 2020-12-15 10:58:55 +01:00
Christian Kamm
c9dbe46542
Checksums: Fix crash due to threading issue
The checksum computation thread was potentially using a QFile that was
deleted in the gui thread.

For #7368
2020-12-15 10:58:54 +01:00
Christian Kamm
a587cd3a13
AccountSettings: Fetch subitems after wiping them
That helps avoid empty lists after account creation #7336
2020-12-15 10:58:54 +01:00
Christian Kamm
5761f4cd8a
SyncEngine: Don't duplicate fatal errors
Previously fatal error texts were duplicated: Once they entered the
SyncResult via the SyncFileItem and once via syncError().

syncError is intended for folder-wide sync issues that are not pinned
to particular files. Thus that duplicated path is removed.

For #5088
2020-12-15 10:58:54 +01:00
Christian Kamm
61972c35a8
Discovery: Parse etag to be consistent with RequestEtagJob
This avoids unnecessary sync runs.

For #7345
2020-12-15 10:58:54 +01:00
Christian Kamm
38c466efbf
Vfs: Require local discovery after disabling vfs
Without it local files aren't guaranteed to be downloaded #6936.
2020-12-15 10:58:54 +01:00
Christian Kamm
55ee3f440b
Vfs: Remove old db record when dehydrating via rename
For #7338
2020-12-15 10:58:54 +01:00
Christian Kamm
e6990bd04e
Vfs: Make move detection work with virtual files #7001
Previously a checksum computation could be done on a suffix-placeholder
file, making discovery believe that no move took place.
2020-12-15 10:58:54 +01:00
Christian Kamm
00dcf3ef59
Vfs: Ensure pins change with (de-)hydration
Previously an implicit hydration of a file in an online-only folder
would not change the pin state and cause a dehydration on the next
sync.
2020-12-15 10:58:53 +01:00
Christian Kamm
0df3b83bd2
Vfs: Add hook to allow update-metadata for unchanged files
Allows winvfs to convert files to placeholders when vfs is enabled. This
is required to mark files as in-sync #7329.
2020-12-15 10:58:53 +01:00
Christian Kamm
b34df4413b
Vfs: Improve strings for availability states 2020-12-15 10:58:53 +01:00
Christian Kamm
5bc2180478
Vfs: Preserve pin state on hydration
For #7322 and #7323
2020-12-15 10:58:53 +01:00
Christian Kamm
fc52c5d0cd
Vfs: Retain existing data when enabling vfs #7302
Previously all local data was deleted because the root folder was marked
as OnlineOnly.
2020-12-15 10:58:53 +01:00
Dominik Schmidt
92f6d866e1
Add missing OWNCLOUDSYNC_EXPORT 2020-12-15 10:58:52 +01:00
Christian Kamm
5acb157a7e
VfsSuffix: Wipe stale pin states #7273
Previously the pin states of deleted files stayed in the 'flags'
database and could be inadvertently reused when a new file with the same
name appeared. Now they are deleted.

To make this work right, the meaning of the 'path' column in the 'flags'
table was changed: Previously it never had the .owncloud file suffix.
Now it's the same as in metadata.path.

This takes the safe parts from #7274 for inclusion in 2.6. The more
elaborate database schema changes (why use 'path' the join the two
tables in the first place?) shall go into master.
2020-12-15 10:58:52 +01:00
Christian Kamm
0e9f030b0f
RequestEtagJob: Consistently parse etags #7271
Previously RequestEtagJob did return the etag verbatim (including extra
quotes) while the db had the parsed form. That caused the etag
comparison during discovery move detection to always fail. The test
didn't catch it because the etags there didn't have quotes.

Now:
- RequestEtagJob will parse the etag, leading to a consistent format
- Tests have etags with quotes, detecting the problem
2020-12-15 10:58:52 +01:00
Christian Kamm
69fa1e4775
UploadDevice: Fix windows issues #7264
- Close the UploadDevice to close the QFile after the PUT job is done.
  This allows winvfs to get an oplock on the file later.

- Don't rely on QFile::fileName() to be valid after
  openAndSeekFileSharedRead() was called. The way it is openend on
  Windows makes it have an empty filename.
2020-12-15 10:58:52 +01:00
Christian Kamm
fb47419e84
Vfs: Move pin state if files move #7250
Previously renames of items didn't carry the pin state with them.
2020-12-15 10:58:52 +01:00
Christian Kamm
34dc5e4e88
Vfs: Don't let new local files start out unpinned #7250
If one adds a new file to an online-only folder the previous behavior
was to upload the file in one sync and dehydrate it in the next. Now
these new files get set to Unspecified pin state, making them retain
their data.
2020-12-15 10:58:52 +01:00
Markus Goetz
62d876b09a
OAuth2: Better error logging
This does not fix a bug, just was found while spotting a bug that was no bug.
For https://github.com/owncloud/enterprise/issues/2951
2020-12-15 10:58:51 +01:00
Christian Kamm
452ed56571
Upload: Read file chunks gradually #7226
Instead of all at once, to reduce peak memory use.

Changing UploadDevice in this way requires keeping the file open for the
duration of the upload. It also means changes to open(), seek(), close()
to ensure that uses of the device work right when a request needs to
be resent.
2020-12-15 10:58:51 +01:00
Christian Kamm
a7852e3aba
Http2: Resend requests on ContentReSend error #7174
Since Qt does not yet transparently resend HTTP2 requests in some cases
we do it manually.

The test showed a problem where the initial non-200 reply would close
the target temporary file and the follow-up request couldn't store any
data. Removing that close() call is safe because there also is a
_saveBodyToFile flag that guards writes to the target file.
2020-12-15 10:58:51 +01:00
Olivier Goffart
c335f69a65
Discovery: Do not abort the sync in case of error 404 (or 500)
Issue: #7199
2020-12-15 10:58:51 +01:00
Olivier Goffart
9f08636a4a
SyncEngine: Fix renaming a single file cause the "delete all file" popup
Possibly a regression, since the new discovery discovers rist the renamed
files as removed

Issue #7204
2020-12-15 10:58:51 +01:00
Olivier Goffart
2c975b1e70
SocketAPI: Fix string claiming a folder is a file
Issue #7206
2020-12-15 10:58:51 +01:00
Markus Goetz
46c336503a
Log Window: Create and open folder properly #7166 2020-12-15 10:58:50 +01:00
Christian Kamm
ee1078b768
owncloudcmd: log sync errors
Previously it was hard to debug some errors since the messages would not
show up in the output.
2020-12-15 10:58:50 +01:00
Christian Kamm
edad7ce7d7
Vfs: Mark sqlite temporaries excluded on db-open #7141
The previous patch ensured that the sqlite temporaries weren't deleted
and recreated for every sync run, but there was still time between
client startup and the first sync run where they would have the
"needs-sync" icon.
2020-12-15 10:58:50 +01:00
Christian Kamm
7774b8049e
Vfs: Distinguish availability error kinds #7143
Previously "no-availability" meant db-error and querying the
availability of a nonexistant path returned AllHydrated.

Now, the availability has a DbError and a NoSuchItem error case.
2020-12-15 10:58:49 +01:00
Christian Kamm
1e5ae77994
Fix logic for duration that an etag reply indicates connectivity
This got inverted accidentally when std::chrono was introduced.

For #7160
2020-12-15 10:58:49 +01:00
Christian Kamm
7f3f13fd97
Vfs: "free space" only shows when it has an effect #7143
To do this, introduce AllDehydrated availability and rename
SomeDehydrated to Mixed - it now guarantees there are also hydrated
items.
2020-12-15 10:58:49 +01:00
Christian Kamm
40d9fc4f4b
Vfs: Adjust and centralise action text #7143
Saying "Currently available locally" sounds more like an indicator than
"Availably locally" does. Centralizing translations avoids consistency
issues between shell context menus and sync folder context menu.
2020-12-15 10:58:49 +01:00
Christian Kamm
021f994584
FolderWizard: Don't crash when typing invalid drive #7041
When the user typed "x:" where the drive x didn't exist, the validation
function would loop forever. Now it shows a "path doesn't exist" error.
2020-12-15 10:58:49 +01:00
Christian Kamm
21cb93e3ce
SyncEngine: Don't close db when done #7141
The db-close operation is likely a leftover from when the SyncEngine
owned its own db connection and serves no purpose anymore.

Closing the db causes the removal of the temporary wal and shm files.
These files are recreated when the db is opened again, which happens
almost immediately.

This is a problem for winvfs because the delete-recreate step wipes the
exclusion state on these files just after the sync is done. That meant
that the db temporaries permanently had a "needs sync" icon marker shown
in the explorer.

Avoiding reopening the db also reduces the number of log messages per
sync.
2020-12-15 10:58:48 +01:00
Christian Kamm
a4f357ee4b
Sqlite: Use FULL synchronous mode with non-WAL journal
According to the documentation DELETE+NORMAL isn't safe from corruption
on older file systems.
2020-12-15 10:58:48 +01:00
Christian Kamm
076e24f465
Log: Start logging immediately when "permanent logs" enabled #7146
Previously one would need to wait for the next sync run to create the
first log file.
2020-12-15 10:58:48 +01:00
Christian Kamm
772a210cc9
Mention selective sync when switching on vfs 2020-12-15 10:58:48 +01:00
Christian Kamm
cfbcdc01db
Discovery: Improvements to doc comments 2020-12-15 10:58:48 +01:00
Christian Kamm
fbe2dbf4ab
Discovery: Query data-fingerprint on root item
Previously the property wasn't queried, meaning the fingerprint logic
couldn't get triggered.
2020-12-15 10:58:48 +01:00
Christian Kamm
c50f041c5b
Discovery: 403 and 503 on root cause error
Previously these result codes during remote discovery of the sync root
would not cause an error and the discovery would get stuck.

Also extends RemoteDiscovery tests to check for errors on the root item.
2020-12-15 10:58:48 +01:00
Christian Kamm
7b96321df2
PropagateIgnore: Default to NormalError for INSTRUCTION_ERROR
Previously if one set the instruction to ERROR while forgetting to set
an error status, it'd propagate as FileIgnored. Now the default is
NormalError for INSTRUCTION_ERROR and FileIgnored for
INSTRUCTION_IGNORE.
2020-12-15 10:58:47 +01:00
Christian Kamm
da40e84aec
About: Add remark about vfs plugin that's in use #7137 2020-12-15 10:58:47 +01:00
Christian Kamm
2bffde2600
Fix windows build 2020-12-15 10:58:47 +01:00
Christian Kamm
22234e0e73
LogWindow: Remove output, add "go to log folder" button #6475 2020-12-15 10:58:47 +01:00
Christian Kamm
dcf34316fd
Vfs: Add 'availability', a simplified, user-facing pin state #7111
The idea is that the user's question is "is this folder's data available
offline?" and not "does this folder have AlwaysLocal pin state?".
The the answers to the two questions can differ: an always-local
folder can have subitems that are not always-local and are dehydrated.

The new availability enum intends to describe the answer to the user's
actual question and can be derived from pin states. If pin states aren't
stored in the database the way of calculating availability will depend
on the vfs plugin.
2020-12-15 10:58:47 +01:00
Christian Kamm
590db28541
Vfs: Clear up relationship between _type and pin state
The pin state is a per-item attribute that has an effect on _type:
AlwaysLocal dehydrated files will be marked for hydration and OnlineOnly
hydrated files will be marked for dehydration.

Where exactly this effect materializes depends on how the pin states are
stored. If they're stored in the db (suffix) the dbEntry._type is
changed during the discovery.

If the pin state is stored in the filesystem, the localEntry._type must
be adjusted by the plugin's stat callback.

This patch makes pin states behave more consistently between plugins.
Previously with suffix-vfs pin states only had an effect on new remote
files. Now the effect of pinning or unpinning files or directories is as
documented and similar to other plugins.
2020-12-15 10:58:46 +01:00
Christian Kamm
2738f110f2
Sqlite: Update to 3.27.2 2020-12-15 10:58:46 +01:00
Christian Kamm
cd10e3d28c
PropagateDirectory: Set initial dir mtime to server mtime #7119
It's still not synced in any way later.
2020-12-15 10:58:46 +01:00
Christian Kamm
69887c531e
PropagateDirectory: Remove dead code
1. The _firstJob is usually deleted by the time the PropagateDirectory
   finishes. (deleteLater() is called early)
2. The PropagateDirectory::_item and PropagateRemoteMkdir::_item point
   to the same SyncFileItem anyway. This code is a leftover from when
   each job had its own instance.
2020-12-15 10:58:46 +01:00
Christian Kamm
fd9b01981b
Detect missing server data during discovery #7112
This has two positive effects:
- We can put the error on the particular file that has missing data
- We can sync all other files
2020-12-15 10:58:46 +01:00
Christian Kamm
4bab93b246
Vfs: Better handling and more tests for suffix file renames
Previously removing the vfs suffix of a file always triggered a
conflict. Now it may just cause a file download.

This was done because users expected symmetry in the rename actions and
renaming foo -> foo.owncloud already triggers the "make the file
virtual" action. Now foo.owncloud -> foo triggers the "download the
contents" action.
2020-12-15 10:58:46 +01:00
Christian Kamm
5e5b0b3f76
Vfs suffix: Require suffix when creating placeholder files 2020-12-15 10:58:45 +01:00
Christian Kamm
3f55f9302e
Vfs: Hydrating a virtual is SYNC not NEW #7101
Previously it'd be NEW(ItemTypeFile), but now it has changed to be
SYNC(ItemTypeVirtualFileDownload) which allows better classification.
2020-12-15 10:58:45 +01:00
Christian Kamm
e0c8acc195
Notifications: Don't say "downloaded" for new files #7101
These files may very well just be new virtual files that were explicitly
*not* downloaded.
2020-12-15 10:58:45 +01:00
Christian Kamm
0c0049c908
Local discovery tracking: On success, also wipe _renameTarget
It's possible that the rename target was in the local discovery list.
2020-12-15 10:58:45 +01:00
Christian Kamm
04e0e22513
Protocol, Notifications: Show destination() instead of _file
destination() now consistently points to the file after the successful
sync operation. _file might be the place the item was moved from.
2020-12-15 10:58:45 +01:00
Christian Kamm
47f10fbf96
Fix and test _file and _renameTarget
There was a bunch of inconsistency around whether _file was set to
_renameTarget or not. This is now never done, passing on more
information.
2020-12-15 10:58:44 +01:00
Christian Kamm
0cf19123a7
VFS: Unbreak behavior for rename+hydrate #7001
Users can rename a file *and* add/remove the vfs suffix at the same time
leading to very complex sync actions. This patch doesn't add support for
them, but adds tests and makes sure these cases do not cause unintened
behavior.

The rename will be propagated, but the users's hydrate/dehydrate request
will be ignored.
2020-12-15 10:58:44 +01:00
Olivier Goffart
4346567a03
Async Poll: keep the size in the database
This was not required with 2.5 because a size of 0 was ignorted when comparing
size by the csync updater, to be compatible with very old version of the database.
But the we discovery will still think the file is changed if the database contains
a size of 0
2020-12-15 10:58:44 +01:00
Olivier Goffart
46bf3ed31a
Upload: asynchronious operations
Implements https://github.com/owncloud/core/pull/31851
2020-12-15 10:58:44 +01:00
Markus Goetz
32c60c2f5d
macOS: Fix vfs suffix plugin paths #7090 2020-12-15 10:58:44 +01:00
Christian Kamm
d496aa5933
Linux: Add autostart delay to avoid tray issues #6518
It seems that sometimes the tray implementation isn't ready on system
startup. Retrying later seems to not help. Delaying the start of the
client is the workaround that people have reported as effective.
2020-12-15 10:58:43 +01:00
Markus Goetz
5da48a5239
SocketApi: Fix owncloud/enterprise#2938 2020-12-15 10:58:43 +01:00
Christian Kamm
2f7cdb81cf
Tray: Try to establish tray after 10s if failed initially #6518
When owncloud is started during desktop startup the tray may not yet
be running when the client starts. This will make the client attempt
to create a tray icon again after 10 seconds if there's no tray
during initial startup.
2020-12-15 10:58:43 +01:00
Christian Kamm
4bd062f5be
OwnSql: Distinguish no-data from error #6677
This could fix a problem where the client incorrectly decides to delete
local data.

Previously any sqlite3_step() return value that wasn't SQLITE_ROW would
be interpreted as "there's no more data here". Thus an sqlite error at a
bad time could cause the remote discovery to fail to read an unchanged
subtree from the database. These files would then be deleted locally.

With this change sqlite errors from sqlite3_step are detected and
logged. For the particular case of SyncJournalDb::getFilesBelowPath()
the error will now be propagated and the sync run will fail instead of
performing spurious deletes.

Note that many other database functions still don't distinguish
not-found from error cases. Most of them won't have as severe effects on
affected sync runs though.
2020-12-15 10:58:43 +01:00
Christian Kamm
ee6a48b3dc
Client certs: Store pkcs12 in config, password in keychain
It still reads and writes the old format too, but all newly stored
client certs will be in the new form.

For #6776 because Windows limits credential data to 512 bytes in older
versions.
2020-12-15 10:58:43 +01:00
Christian Kamm
cc840534c0
Add PLUGINDIR cmake setting and define #7027
By default, plugins are only searched next to the binary or next to the
other Qt plugins. This optional build variable allows another path to be
configured.

The idea is that on linux the oC packaging probably wants the binary in
something like /opt/owncloud/bin and the plugins in
/opt/owncloud/lib/plugins.

Similarly, distribution packagers probably don't want the plugins next
to the binary or next to the other Qt plugins. This flag allows them to
configure another path that the executable will look in.
2020-12-15 10:58:43 +01:00
Olivier Goffart
87bd26bf6c
AccountManager: load the cookies
For issue #7054
2020-12-15 10:58:43 +01:00
Christian Kamm
575935ded0
Windows: Forbid chars 0-31 in filenames #6987 2020-12-15 10:58:42 +01:00
Christian Kamm
a72bf89779
Selective sync: Don't collapse tree when entering mode #7055
doExpand() is called when the selective sync editing mode is enabled in
the folder settings view. Previously it'd set the expansion to be
exactly the root items. Now, it just expands any root items that are
currently collapsed, leaving all other item expansion unchanged.
2020-12-15 10:58:42 +01:00
Christian Kamm
2e11f14a6b
SocketApi: Fix crash with readyRead() after disconnected() #7044
With the recent bugfix to avoid sending messages on dead connections
0bfe7ac250c54f5415c0a794c7b271428e83c3cf
the client now crashed if readyRead() was received after disconnected()
for the socket as the listener for that connection was already removed.

This code fixes it by still invoking the handler from readyRead() but
passing a SocketListener that won't attempt to send messages.
2020-12-15 10:58:42 +01:00
Christian Kamm
6199e140a5
Wizard: Avoid cert dialog multiple connection
This could lead to odd behavior when slotCertificateAccepted
was called multiple times.
2020-12-15 10:58:42 +01:00
Christian Kamm
83268c255a
Folder wizard: Fix infinite loop for bad paths #7041 2020-12-15 10:58:42 +01:00
Christian Kamm
93afc2a04b
Discovery win: Fix detection of case-only renames
Previously they were detected as DELETE+NEW because if "a" is renamed to
"A" then QFile::exists("a") will still return true on Windows.
2020-12-15 10:58:42 +01:00
Christian Kamm
238ac53666
Ensure local discovery on selective sync changes
As far as I'm aware local discovery can be skipped on folders that are
selective-sync blacklisted, so a local discovery is required when an
entry is removed from the blacklist.

Also rename
avoidReadFromDbOnNextSync() -> schedulePathForRemoteDiscovery()
since the old name might also imply it's not read from db in the local
discovery - which is not the case. Use Folder::
schedulePathForLocalDiscovery() for that.
2020-12-15 10:58:41 +01:00
Christian Kamm
4c04351360
Fix warnings about signedness
Sizes are always qint64, not unsigned.
TransferIds are always uint.
2020-12-15 10:58:41 +01:00
Christian Kamm
910ccaf600
Vfs: Improve sync protocol entries for actions
Creating a new virtual file and replacing a file with a virtual one now
have their own text in the protocol, not just "Downloaded".

To do this, the SyncFileItem type is kept as
ItemTypeVirtualFileDehydration for these actions. Added new code to
ensure the type isn't written to the database.

While looking at this, I've also added documentation on SyncFileItem's
_file, _renameTarget, _originalFile and destination() because some of
the semantics weren't clear.
2020-12-15 10:58:41 +01:00
Christian Kamm
f24687ec93
Vfs: Make dehydration a SYNC and not a NEW action
That change will be useful for the notifications. Previously the
dehydrated files were reported as "newly downloaded", now they're
reported as "updated".
2020-12-15 10:58:40 +01:00
Christian Kamm
797734870f
PropagateDownload: Create conflict even if local file changed
Fixes a bug introduced while moving the attribute propagation before the
conflict-renaming.
2020-12-15 10:58:40 +01:00
Christian Kamm
8a8e93827f
Vfs suffix: Fix dehydration creating the wrong db entry 2020-12-15 10:58:40 +01:00
Christian Kamm
b06f67baf0
Vfs: Ensure VfsOff works without start() being called
In tests an un-started Vfs instance is sometimes passed to SyncEngine
via SyncOptions.
2020-12-15 10:58:40 +01:00
Christian Kamm
6a977edeee
Vfs: Remove VfsDefaults
That just complicated things. It's ok if Vfs is not a fully abstract
interface class.

The pinstate-in-db methods are instead provided directly on Vfs and
VfsSuffix and VfsOff use them to implement pin states.

The start() method is simply non-virtual and calls into startImpl() for
the plugin-specific startup code.
2020-12-15 10:58:40 +01:00
Christian Kamm
e9cbe13598
Vfs: Add option to hide socketapi pin actions
Because some plugins provide alternative ui.
2020-12-15 10:58:40 +01:00
Christian Kamm
e6ee5d0f8a
PropagateDownload: Conflict-rename later
The block of code that propagated attributes etc from the previously
existing file was placed *after* the block that renamed the previously
existing file to a conflict name. That meant the propagation didn't work
in the conflict case.
2020-12-15 10:58:39 +01:00
Christian Kamm
6c5fa1dadd
Vfs: dehydration is separate action
Allows for better attribute preservation.

Also add verifyFileUnchanged() call before dehydration to avoid data
loss when discovery takes a while.
2020-12-15 10:58:39 +01:00
Christian Kamm
d8873c18a1
File watcher: Pin state attribute changes are valid notifications
Previously they would be discarded since the file's mtime or size hadn't
changed.
2020-12-15 10:58:39 +01:00
Christian Kamm
83a818678f
PinStates cleanup
- SyncJournalDB functions now behind internalPinStates() to avoid
accidental usage, when nearly everyone should go through Vfs.
- Rename Vfs::getPinState() to Vfs::pinState()
2020-12-15 10:58:39 +01:00
Christian Kamm
0eb4065197
Folder: Add remoteFolderTrailingSlash()
There were cases where the "/" exception wasn't handled correctly
and there'd be extra slashes in generated paths.
2020-12-15 10:58:39 +01:00
Christian Kamm
41f1ddb5fc
Vfs: Call unregisterFolder() when folder is removed 2020-12-15 10:58:39 +01:00
Christian Kamm
597cc60f52
Vfs: Enable propagating attributes on download 2020-12-15 10:58:38 +01:00
Christian Kamm
01f7bc7b49
SyncFileStatusTracker: Distinguish Warning and Excluded
Any folder with a (potentially deeply) contained error will have
StatusWarning. StatusExcluded marks exclusions. The difference is useful
to know for VFS.
2020-12-15 10:58:38 +01:00
Christian Kamm
7f400e3226
Pin state updates
- unspecified and inherited are different
- move enum to header in common/
- access through Vfs instead of directly in Journal
2020-12-15 10:58:38 +01:00
Christian Kamm
2722c61515
Vfs: Shell integration is enabled optionally
Needs to be disabled for tests in some cases.
2020-12-15 10:58:38 +01:00
Christian Kamm
f89450648f
Vfs: Clarify SyncEngine::wipeVirtualFiles()
Possibly the behavior should actually change and the function should
de-placeholder all items, not just dehydrated ones.
2020-12-15 10:58:38 +01:00
Christian Kamm
5820ac8b41
Discovery: Files can have dehydrate/download actions
This will be used in conjunction with vfs plugins that detect whether a
file has a pending hydration/dehydration through independent means and
communicate that to the discovery through local file type.
2020-12-15 10:58:38 +01:00
Christian Kamm
af1666788e
Discovery: Add signal for silentlyExcluded files
This allows SyncFileStatusTracker to also know about these. After all
its information is used to provide icons for them too.
2020-12-15 10:58:37 +01:00
Christian Kamm
e2eea24a03
Vfs: Send SyncFileStatusTracker data to vfs plugins 2020-12-15 10:58:37 +01:00
Christian Kamm
31394f14b5
Vfs: Make files that end up in db placeholders
Since 'placeholder' just means that it's an item of the special type
that the vfs plugin can deal with - no matter whether hydrated or
dehydrated - all done items should become placeholders. Even
directories.

Now every file that passes through updateMetadata() will be converted to
a placeholder if necessary.
2020-12-15 10:58:37 +01:00
Christian Kamm
ca0323e1e7
Propagator: Helper for updating db
Similar steps were done in many propagation jobs.

This also updates the db entry to always have the item.destination() as
file path.
2020-12-15 10:58:37 +01:00
Christian Kamm
fe27804afb
Move SyncFileStatus to libcommon
It'll be needed in vfs plugins so they can connect to the data coming
out of SyncFileStatusTracker.
2020-12-15 10:58:37 +01:00
Christian Kamm
1cc41427fc
Build fix for mingw64 7.3 (Qt 5.12.0) 2020-12-15 10:58:37 +01:00
Christian Kamm
4d58208676
File watcher: Reduce touch ignore duration
On Linux and Windows the file watcher can't distinguish between changes
that were caused by the process itself, like during a sync operation,
and external changes. To work around that the client keeps a list of
files it has touched and blocks notifications on these files for a bit.

The duration of this block was originally and arbitrarily set at 15
seconds. During manual tests I regularly thought there was a bug when
syncs didn't trigger, when the only problem was that my changes happened
too close to a previous sync operation.

This change reduces the duration to three seconds. I imagine that this
is still enough.

Also use std::chrono while at it.
2020-12-15 10:58:36 +01:00
Dominik Schmidt
539cef345e
Switch to standard opt-out BUILD_TESTING instead of opt-in UNIT_TESTING
Compare https://cmake.org/cmake/help/v3.0/module/CTest.html
Craft automatically handles BUILD_TESTING, so we don't need to handle it
in our own blueprint.
2020-12-15 10:58:36 +01:00
Christian Kamm
13890c04a7
HttpCreds: Warn in log if keychain-write jobs fail #6776
Also, calling deleteLater() on jobs is unnecessary (they autodelete
after finished()) and deleting the attached QSettings is also
unnecessary because the settings object is parented to the job.
2020-12-15 10:58:36 +01:00
Christian Kamm
1a250bc3c7
Download: Ignore content-length for compressed HTTP2/SPDY replies #6885
It contains the compressed size.

See https://bugreports.qt.io/browse/QTBUG-73364
2020-12-15 10:58:36 +01:00
Christian Kamm
848b869cdf
SocketAPI: Remove listener immediately on lost connection
To avoid situations where messages are attempted to be sent to dead
connections.
2020-12-15 10:58:36 +01:00
Christian Kamm
0eebd77d2c
SocketAPI OSX: Forbid further sends on connectionDidDie
Also release the remote end immediately.
2020-12-15 10:58:36 +01:00
Olivier Goffart
6f4bf585f0
Move: Fix move detection in directory move on the other side 2020-12-15 10:58:35 +01:00
Olivier Goffart
88d02a887f
Move: add comments and tests 2020-12-15 10:58:35 +01:00
Olivier Goffart
1fb4c22adf
Move: add more test and fix move within moves 2020-12-15 10:58:35 +01:00
Olivier Goffart
ade4c11de3
Rename: fix renamed folder moved into renamed folder issue
Issue #6694
2020-12-15 10:58:35 +01:00
Christian Kamm
f3be99e5b2
vfs: Update pinning context menu to be less confusing
Seeing "Currently available online only" for a currently hydrated file
was odd. It makes sense since current hydration status and pin state are
independent.

The new text will say something like "Currently available, but marked
online only" to better indicate that the file might be dehydrated later
since it wasn't pinned.
2020-12-15 10:58:35 +01:00
Christian Kamm
b7079289c2
Vfs: Switch on/off only when sync isn't running #6936
Avoids some situations that might cause data loss.
2020-12-15 10:58:34 +01:00
Christian Kamm
5728256763
Folder: Add selective sync / ui related flags
supportsSelectiveSync(): clearer than !supportsVirtualFiles() and allows
  extra logic

isVfsOnOffSwitchPending(): Somewhat awkward way of dealing with the
  phase between a user requesting vfs state to be switched and it
  actually happening
2020-12-15 10:58:34 +01:00
Christian Kamm
b91839b760
SyncEngine: Rename phase finishing slots
slotDiscoveryJobFinished -> slotDiscoveryFinished
slotFinished -> slotPropagationFinished

This should be clearer. Particular the
slotFinished -> finalize -> emit finished()
chain was confusing before.
2020-12-15 10:58:34 +01:00
Christian Kamm
9f8505c736
Discovery: Fix aborting during discovery #6972 2020-12-15 10:58:34 +01:00
Christian Kamm
836d298141
vfs suffix: Ignore server files or synced files with the suffix #6953 2020-12-15 10:58:34 +01:00
Christian Kamm
e774c6c739
Discovery: Ensure selective sync lists are sorted #6958 2020-12-15 10:58:34 +01:00
Christian Kamm
22255e4976
vfs: Add vfs migration options to folder context menu
This allows enabling and disabling vfs.

To distinguish this operation from setting the root pin state, the
availability setting is adjusted as well to be similar to the
menu that shows in the shell extensions.
2020-12-15 10:58:33 +01:00
Christian Kamm
20ef0a0290
vfs: Allow retrieving of pin state paths and flags 2020-12-15 10:58:33 +01:00
Christian Kamm
7ef6e60660
Result: Add copy/move ctor/op=
It has a destructor and these operations make sense. Particularly the
move is important for code like:

Result<x, y> foo() { Result<x, y> v; return v; }

because the move-ctor will not autogenerate if x or y are not trivially
destructible.
2020-12-15 10:58:33 +01:00
Christian Kamm
3b923e2afe
vfs: Change per-folder message on vfs support
I'm unsure this note is all that useful. Remove entirely?
2020-12-15 10:58:33 +01:00
Christian Kamm
4df101ed84
vfs: Allow (de-)hydrating the full sync folder 2020-12-15 10:58:33 +01:00
Christian Kamm
47e21bfc60
vfs: Don't assume suffix mode for old folders
That would break with old folders that use selective sync.
2020-12-15 10:58:33 +01:00
Christian Kamm
cbb32edee2
vfs: Disable selective sync if vfs support is available 2020-12-15 10:58:32 +01:00
Christian Kamm
aa23058d76
Wizard: Ensure client cert doesn't get lost #6911 2020-12-15 10:58:32 +01:00
Christian Kamm
c470825c8f
Wizard: Drop unused function 2020-12-15 10:58:32 +01:00
Markus Goetz
4fcad0d8b0
Sync: Display theme in debug log 2020-12-15 10:58:32 +01:00
Christian Kamm
b4e73690b7
sqlite: Set exclusive locking_mode to avoid WAL issues #6881
Can be overridden with OWNCLOUD_SQLITE_LOCKING_MODE
2020-12-15 10:58:32 +01:00
Dominik Schmidt
17465a6f94
Fix lookup of system wide settings 2020-12-15 10:58:32 +01:00
Olivier Goffart
f258af1198
sqlite: Update bundled version to 3.26.0
For OS X and Windows.
2020-12-15 10:58:31 +01:00
Olivier Goffart
c31d3f277f
HttpCredentials: Do not re-enter the event loop
https://sentry.io/owncloud/desktop-win-and-mac/issues/777907931/
mention a crash in OCC::HttpCredentialsGui::showDialog
One possible explaination is that this is caused by re-entring the event loop.
So don't do that.
2020-12-15 10:58:31 +01:00
Olivier Goffart
51d2e41d8b
Chunking Move: Fix too many starting slashes in the destination header
Commit 6ca724f fixed it for the move case. But the upload MOVE did the same

Issue #6904
2020-12-15 10:58:31 +01:00
Olivier Goffart
0e56dfe3a4
Gui: do not show the settings when opening a virtual file
Issue #6764
2020-12-15 10:58:31 +01:00
Christian Kamm
c4dfa82661
SocketAPI: Overhaul ui for 'available locally' vs 'online only'
For #6815
2020-12-15 10:58:30 +01:00
Christian Kamm
aa382eda29
Db: Add wiping of pin state for subtrees 2020-12-15 10:58:30 +01:00
Christian Kamm
d956f518a8
vfs: Remove newFilesAreVirtual - use root PinState instead
This unifies how to deal with pin states.

Also enable reading a folders direct pin state vs its effective pin
state.
2020-12-15 10:58:30 +01:00
Olivier Goffart
62ec4c9330
Discovery: Handle the blacklistFiles from the server capabilities
Issue #434

Ideally one could add the blacklist to the exlucde reggexp, but this
is simpler
2020-12-15 10:58:30 +01:00
Christian Kamm
8fecff5153
Progress: Virtual file creation needs no transfer progress #6933
Treat virtual file creation as a size-less action, similar to
propagating renames or deletions.
2020-12-15 10:58:30 +01:00
Christian Kamm
cdf61b9e82
vfs: Show "new files are virtual" option only with vfs
If there's no vfs possibility, showing it makes little sense.
2020-12-15 10:58:30 +01:00
Christian Kamm
8f895fc83c
vfs: SocketAPI actions adjust pin state of directories #6815
Downloading a folder also sets its pin state; releasing a folder sets
its pin state.
2020-12-15 10:58:29 +01:00
Christian Kamm
486c25cb47
vfs: Use PinState in sync algorithm #6815
New files are virtual if the file's pin state is OnlineOnly.
2020-12-15 10:58:29 +01:00
Christian Kamm
68126ac208
vfs: Introduce PinState db storage #6815
The idea is to allow folders (and later maybe files?) to be
- pinned to be available locally
- pinned to be online only
- inherit their pin from the parent

Where this pinning only controls the default for new files.
Subfolders may have a different pin state, and contained files
may be hydrated or dehydrated based on user actions.

This value is stored in a new 'flags' table. The idea is to store
data there that doesn't necessarily exist for each metadata entry.
The selective sync state could be migrated to this table.
2020-12-15 10:58:29 +01:00
Olivier Goffart
a29320b18d
Discovery: Set right direction when restoring deleted discovery because it has modified files
(Catched by a faillure of t1.pl)
2020-12-15 10:58:29 +01:00
Olivier Goffart
6da96cd026
Port the exclude test to the QTest Framework
This is just a port to QtTest, I did not change the layout of the test.
I did search and replace to replace the assert with QCOMPARE/QVERIFY
I still call setup and setup_init like before (only explicitly, now)

Also ported the preformence tests to QBENCHMAK because windows don't have
gettimeofday.

Relates #6358
2020-12-15 10:58:29 +01:00
Christian Kamm
69144566ce
Database: Change path for new dbs to .sync_* #5904
This is to avoid issues on OSX, where the ._ prefix has special meaning.

Originally (before 2.3.2) ._ was necessary to guarantee exclusion. But
since then the .sync_ prefix is excluded as well.

This does not affect existing database files.
2020-12-15 10:58:28 +01:00
Christian Kamm
30294e0c9a
Vfs: Have a static list of potential plugins for now
Fixes in-source builds and other cases where more non-plugin directories
are created in src/libsync/vfs.
2020-12-15 10:58:28 +01:00
Christian Kamm
75b9976526
Vfs: Ensure older versions gracefully ignore winvfs folders
Previously there'd likely be a mess if a 2.6 winvfs folder was attempted
to be used with a 2.5 client. Now the older clients will ignore these
folders.
2020-12-15 10:58:28 +01:00
Christian Kamm
bdfda460e6
Vfs plugins: Available plugins must load
A plugin that can't be loaded due to dependency issues should not be
considered as available.
2020-12-15 10:58:28 +01:00
Olivier Goffart
fd410a5a84
SyncEngine: Ensure that the paths passed to the discovery ends with slashes
This was making the tx.pl test fail
2020-12-15 10:58:28 +01:00
Olivier Goffart
edd806960d
Propagator: Don't abort sync on error 503
Only do it when it is actually a maintenance mode

Issues #5088, #5859, https://github.com/owncloud/enterprise/issues/2637
2020-12-15 10:58:27 +01:00
Olivier Goffart
feb770eca7
Allow to open log window via command line to already running client
Issue: #4098
2020-12-15 10:58:27 +01:00
Dominik Schmidt
687eb9665d
Fix installation of vfs suffix plugin 2020-12-15 10:58:27 +01:00
Olivier Goffart
a7a54b6d11
VFS: Plugin needs to be installed, otherwise it can't be loaded 2020-12-15 10:58:27 +01:00
Olivier Goffart
47b2913357
Fix compilation warning
src/libsync/propagatorjobs.cpp:63:10: warning: lambda capture 'this' is not used [-Wunused-lambda-capture]
        [this, &deleted](const QString &path, bool isDir) {
         ^~~~~
2020-12-15 10:58:27 +01:00
Christian Kamm
d87c2b803d
vfs: Make Vfs ctor explicit 2020-12-15 10:58:27 +01:00
Christian Kamm
305d439c41
vfs: Separate vfs availability from new-files-virtual
This helps support 2.5 settings where there are virtual files in the
tree but new files aren't created virtual.

It's also a prelude for #6815

There's currently no way of
- upgrading vfs plugins (a silent suffix->winvfs upgrade is attempted
  once only, when moving to master)
- disabling vfs capabilities outright
2020-12-15 10:58:27 +01:00
Christian Kamm
1e5e884805
vfs: Don't always load plugins, check metadata 2020-12-15 10:58:26 +01:00
Christian Kamm
cc912f4d02
vfs: ensure backwards-compatible settings value is always set 2020-12-15 10:58:26 +01:00
Christian Kamm
b30f79edf6
vfs: Ensure SyncOptions::_vfs is never null
- Create a VfsOff derived class
- Make it a shared pointer shared with Folder::_vfs
2020-12-15 10:58:26 +01:00
Christian Kamm
842577e014
vfs: Switch order of deletion and journal removal in dehydration
To be more crash-resilient.
2020-12-15 10:58:26 +01:00
Christian Kamm
4e40b635da
vfs: Update outdated comment 2020-12-15 10:58:26 +01:00
Christian Kamm
d7ad7854c1
vfs: Ensure local discovery is done on dehydration request 2020-12-15 10:58:26 +01:00
Christian Kamm
f2b78b5efb
Folder settings: Keep "usePlaceholders" to be backwards compatible 2020-12-15 10:58:25 +01:00
Christian Kamm
fa2450cf11
vfs: Be more careful about Vfs instance ownership 2020-12-15 10:58:25 +01:00
Christian Kamm
9196aa8e0a
vfs: Remove VfsSuffixPrivate 2020-12-15 10:58:25 +01:00
Christian Kamm
5b26b739f0
vfs: Document ignored return value. 2020-12-15 10:58:25 +01:00
Christian Kamm
77e5b956d1
Plugin: Drop PluginLoader 2020-12-15 10:58:25 +01:00
Christian Kamm
9bc28e3006
Plugin: style fixes 2020-12-15 10:58:25 +01:00
Christian Kamm
a5264f7740
Checksums: Make file ownership more explicit 2020-12-15 10:58:24 +01:00
Christian Kamm
9ced8dd836
vfs: Improve modeFromString() signature 2020-12-15 10:58:24 +01:00
Christian Kamm
f502a526fa
Generalize Result<> class, add Optional<>
To make it nicer to use outside of HTTP results.
2020-12-15 10:58:24 +01:00
Christian Kamm
e7e6b839c0
vfs: Add API docs 2020-12-15 10:58:24 +01:00
Christian Kamm
85dd10eb9b
vfs: Fix dealing with missing vfs instance
Maybe there should just be a VfsOff instance to make this less error
prone.
2020-12-15 10:58:24 +01:00
Dominik Schmidt
bfe136da7a
Fix export of vfs functions 2020-12-15 10:58:23 +01:00
Dominik Schmidt
1104883fba
Cleanup CMake (output) 2020-12-15 10:58:23 +01:00
Christian Kamm
f074c1a07b
vfs: Add check for presence of win plugin 2020-12-15 10:58:23 +01:00
Christian Kamm
4bef96afe7
vfs: Make switching vfs on/off work again 2020-12-15 10:58:23 +01:00
Christian Kamm
fa6f3cd847
vfs: Fix plugin decision in wizards, sanitize loading 2020-12-15 10:58:23 +01:00
Christian Kamm
d5a3604d39
vfs: Reenable local metadata updating of vfs files 2020-12-15 10:58:23 +01:00
Christian Kamm
96f4fd46e5
vfs: Make some behaviors suffix-vfs specific 2020-12-15 10:58:23 +01:00
Christian Kamm
90827c8162
vfs: Allow remote renames to propagate as such 2020-12-15 10:58:22 +01:00
Christian Kamm
e0ae6012b1
vfs: Allow folders without vfs 2020-12-15 10:58:22 +01:00
Christian Kamm
e39d751b59
Fix big-folder detection when vfs is disabled 2020-12-15 10:58:22 +01:00
Christian Kamm
bee1b0edd4
vfs: Implement dehydration tagging 2020-12-15 10:58:22 +01:00
Christian Kamm
ae9a7e088f
vfs: Fix suffix detection and handling 2020-12-15 10:58:22 +01:00
Christian Kamm
1ed005aafe
winvfs: Better error reporting
- fix download and validation error reporting
- add tests
2020-12-15 10:58:22 +01:00
Christian Kamm
0f2ef42ba2
Fixes after rebase to master
- use vfs suffix in ProcessDirectoryJob
- fix include vfs.h
- fix local vio passing vfs
- fix checksum computation
- vfs mode use
- mingw lambda compile issue
2020-12-15 10:58:21 +01:00
Christian Kamm
2b20985875
winvfs: initial work
Done by ckamm and dschmidt
2020-12-15 10:58:21 +01:00
Christian Kamm
c3b1a872aa
FolderMan: Remove assumption of unique running sync 2020-12-15 10:58:21 +01:00
Christian Kamm
22e08cf6ad
Checksums: Work on QIODevice*s
Needed for cfapi where we want to feed data through a custom device
which retrieves data from the windows api.
2020-12-15 10:58:21 +01:00
Olivier Goffart
a6614c18f1
IgnoreListEditor: increase a bit the size
Should Help for issue #6641
2020-12-15 10:58:21 +01:00
Olivier Goffart
7c1871ae65
Account: Fix crash when deleting an account
We should not have request parented to the account, otherwise we might get
a loop on the deletion order.

Issue #6893
2020-12-15 10:58:21 +01:00
Christian Kamm
6b0bb66b17
LockWatcher: Add chrono header 2020-12-15 10:58:20 +01:00
Christian Kamm
582803b6bd
cjhash: Fix build fix
See 17a4055e8d658f24b1fc47c44fd2daf66c7d5dbf
2020-12-15 10:58:20 +01:00
Olivier Goffart
175b064364
cjhash: use Q_FALLTHROUGH
This fixes the warning
   warning: unknown option after ‘#pragma GCC diagnostic’ kind [-Wpragmas]

Issue #6872
2020-12-15 10:58:20 +01:00
Kevin Ottens
cce3d6f6fc
Excludes: switch all the path handling to QString
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-12-15 10:58:20 +01:00
Olivier Goffart
1c443ad021
CMakeLists.txt: increment sqlite version
3.9 is required for custom index to work
2020-12-15 10:58:20 +01:00
Olivier Goffart
5e05b61d2f
csync_vio_locale: use QString for the path in opendir
So we don't have to convert to utf8 and back again
2020-12-15 10:58:20 +01:00
Olivier Goffart
f0c24cb6f9
Remove csync_misc: The code no longer use fnmatch 2020-12-15 10:58:20 +01:00
Olivier Goffart
3c3619f99a
Exclude: do everything with QString wiuthout converting to char* 2020-12-15 10:58:19 +01:00
Olivier Goffart
164051b0c9
ProcessDirectoryJob::process: optimize so there is only one map 2020-12-15 10:58:19 +01:00
Olivier Goffart
1783db5811
PropagateUpload: Avoid many allocations by using QByteArrayLiteral 2020-12-15 10:58:19 +01:00
Olivier Goffart
d6a0290058
Optimisation: Add a cache SyncJournalDb::mapChecksumType
No need to do two sql query for something that's always the same and
there are very few checksum types
2020-12-15 10:58:19 +01:00
Olivier Goffart
69de2d5180
Sync: optimize by removing setFileRecordMetadata
Inh most case we already have a record from before, so avoid doing a useless
lookup in the database.
In owncloudpropagator.cpp, directories do not have a checksum so no need
to call a function that preserves it
2020-12-15 10:58:19 +01:00
Olivier Goffart
7e1840bb2f
OwnSql: Speedup by avoiding allocating an error string on success 2020-12-15 10:58:18 +01:00
Gerhard Gappmeier
7e55ce6640
owncloudcmd: fix compiliation with TOKEN_AUTH_ONLY 2020-12-15 10:58:18 +01:00
Gerhard Gappmeier
b3792ac1f0
libsync: Fix build error with TOKEN_AUTH_ONLY
When enabling TOKEN_AUTH_ONLY, the code path using QPainter is disabled.
So we also don't need the includes.
This header is not available for Remarkable.
2020-12-15 10:58:18 +01:00
Gerhard Gappmeier
ff9bd84c45
Disable stack protector features when cross compiling
When cross-compiling this for remarkable using the poky toolchain
this results in linker errors with stack protector: libssp_nonshared.a
not found
2020-12-15 10:58:18 +01:00
Gerhard Gappmeier
6aead6425e
Add CMake option to disable GUI build 2020-12-15 10:58:18 +01:00
Kevin Ottens
9131a68ec4
Get DatabaseErrorTest to pass
The E2EE code path would get the engine to go wrong in case of db error.
It's just better to have a failing upload or failing mkdir later in those
cases.

Emitting signals from a ctor is a bad idea anyway

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-12-15 10:58:18 +01:00
Olivier Goffart
bb9ce8db21
Test that the sync behave well if there are errors while reading the database 2020-12-15 10:58:17 +01:00
Olivier Goffart
b79e57d1c1
Discovery: fix double emission of finished in case of error 2020-12-15 10:58:17 +01:00
Olivier Goffart
7061f31887
Discovery: Fix downloading files when database is used for local discovery
This also fix the currently failling LockedFilesTest
2020-12-15 10:58:17 +01:00
Olivier Goffart
0eaa950e9e
Remove csync.cpp: It's only dead code 2020-12-15 10:58:17 +01:00
Christian Kamm
0d49056a13
Add Ctrl-L as log window shortcut
F12 is taken on OSX and there's no other way of showing it.
2020-12-15 10:58:17 +01:00
Christian Kamm
9d55590d10
Test: Add test for locked file tracking and propagation 2020-12-15 10:58:17 +01:00
Christian Kamm
da178c1352
Folder: Treat file unlock similar to external change #6822
For consistent handling of incoming notifications.
2020-12-15 10:58:17 +01:00
Christian Kamm
815e0fee8f
Propagator: Add assert against duplicate done() calls 2020-12-15 10:58:16 +01:00
Olivier Goffart
eb23776f16
Discovery: Fix renaming on windows
buf.type is ItemFileSkip because csync_vio_local_stat does not set this field
2020-12-15 10:58:16 +01:00
Olivier Goffart
beee123c80
New Discovery: Fix trailing slash causing failure on windows 2020-12-15 10:58:16 +01:00
Olivier Goffart
60b17cd128
Folder: update the folder version
Since the new index would crash old version of the client, we need to
upgrade folder version so they do not load in the client 2.5.0
2020-12-15 10:58:15 +01:00
Christian Kamm
7de453d439
Settings: Make FoldersWithPlaceholders group sticky
If virtual files are disabled on a folder it might still have db entries
or local virtual files that would confuse older client versions.
2020-12-15 10:58:15 +01:00
Olivier Goffart
ab85c60205
owncloudcmd: Read the server version from the capabilities
Issue: #6846
2020-12-15 10:58:15 +01:00
Olivier Goffart
3e4486c078
owncloudcmd: fetch the dav user
This is required to get the new endpoint working when the server
uses ldap or that the dav user is not the same as the login.

Issue #6830
2020-12-15 10:58:15 +01:00
Olivier Goffart
ec8c02dad0
Sharing: add the shareWithAdditionalInfo string in autocompletion results
Issue #6749
2020-12-15 10:58:15 +01:00
Olivier Goffart
380d7b8028
Migration from 2.4: fallback to move file by file if directory move failled (#6807)
Migration from 2.4: fallback to move file by file if directory move failed

This can happen if the directory already exist because, say, it was
created by the ownCloud outlook plugin which save its file in the same directory
2020-12-15 10:58:15 +01:00
Olivier Goffart
e20e1d110f
Move: Fix too many starting slashes in the destination header
QDir::cleanPath does not remove starting slashes on windows.
So use account::davUrl which is already cleaned

Issue: #6824
2020-12-15 10:58:15 +01:00
Olivier Goffart
dcfbde2a67
Wizard: Reset the QSslConfiguration before checking the server
Because a previous call with another demain might have set some
config for another server.

Issue #6832
2020-12-15 10:58:14 +01:00
Olivier Goffart
b820e46805
Fix compiler warning 2020-12-15 10:58:14 +01:00
Olivier Goffart
35967fc2d6
OAuth2: Refresh the token without aborting the sync
OAuth2 access token typically only has a token valid for 1 hour.
Before this patch, when the token was timing out during the sync, the
sync was aborted, and the ConnectionValidator was then requesting a new
token, so the sync can be started over.
If the discovery takes longer than the oauth2 validity, this means that
the sync can never proceed, as it would be always restarted from scratch.

With this patch, we try to transparently renew the OAuth2 token and restart
the jobs that failed because the access token was invalid.

Note that some changes were required in the GETFile job because it handled
the error itself and so it was erroring the jobs before its too late.

Issue #6814
2020-12-15 10:58:14 +01:00
Olivier Goffart
53a14c2041
HttpCredentials: initialize all member inline 2020-12-15 10:58:14 +01:00
Dominik Schmidt
5d9370594d
Disable autostartCheckBox if autostart is configured system wide 2020-12-15 10:58:14 +01:00
Christian Kamm
00edcf98a1
Discovery: Virtual file handling adjustments
- adjust virtual file path handing
- helpers for vfs suffix adding/removal
- helpers for isDirectory/isVirtual on SyncJournalRecords
- be clear about what PathTuple _local/_server mean
2020-12-15 10:58:14 +01:00
Olivier Goffart
58eaf9940a
Database: Add an index on the parent path
So we can quickly query the items in a parent directory

This uses a custom slite3 function, which means that when downgrading the client,
or using another tool to add entries in the database, any insertion in the metadata
table will produce an error: "unknown function: parent_hash()"
(This will crash the client 2.5)
2020-12-15 10:58:13 +01:00
Olivier Goffart
e45e57982d
Add missing export 2020-12-15 10:58:13 +01:00
Christian Kamm
b431f4ef0c
Discovery: Fix log output for instructions 2020-12-15 10:58:13 +01:00
Christian Kamm
ce420d77a8
Discovery: Don't rebuild invalidFilname regex each call 2020-12-15 10:58:13 +01:00
Christian Kamm
113124cde5
Discovery: Introduce smaller functions 2020-12-15 10:58:13 +01:00
Christian Kamm
4f6f706f40
Excludes: drop csyncTraversalMatchFun()
The new discovery can call the traversal match function directly.
2020-12-15 10:58:13 +01:00
Olivier Goffart
a4e139969c
Fix windows build 2020-12-15 10:58:12 +01:00
Olivier Goffart
8490664860
Account Settings: fix progress being written in white when there are errors 2020-12-15 10:58:12 +01:00
Christian Kamm
dd34cbc751
Virtual files: Wipe selective sync settings when enabled 2020-12-15 10:58:12 +01:00
Christian Kamm
60de1c9720
virtual files: show option only when branding allows it 2020-12-15 10:58:12 +01:00
Christian Kamm
251e01a440
Virtual files: Use theme to check for option availability 2020-12-15 10:58:12 +01:00
Olivier Goffart
94e63ef7b9
Account Settings: Add a context menu entry to enable or disable virtual files
Issue #6725
2020-12-15 10:58:12 +01:00
Olivier Goffart
18f6e346b8
Upload: Do not error out if the server do not send the X-OC-MTime: accepted header
If the server does not set the mtime, it is not a big problem for the
synchronisation.

The test was used before so we could do a PROPPATCH for server that did not
support this header. But now that all server supports that we don't need to
to the check. (We do not do the PROPPATCH since we got rid of the neon
dependency)

Apparently, it may happen that some backend don't support setting mtime
and this can lead to this error.

https://github.com/owncloud/client/issues/6797
2020-12-15 10:58:11 +01:00
Olivier Goffart
15eab07866
OAuth2: Store 'Account::davUser' in the config, and use that user for connecting
We need to use the user id to check if we are connected to the right account.
These might be different from the HTTP Basic Auth login. (LDAP setups)

When the account was configured as an oauth2 account form the wisard, the
http_user was already set correctly to the user id. But when the server is
upgrading from basic auth to oauth2, we need to pick the right login.

Note that Account::davUser() already defaults to the HTTP user when none
is set, so this means the upgrade will be fine if this is not set in the
config.

Issues:
https://github.com/owncloud/oauth2/issues/109
https://github.com/owncloud/enterprise/issues/2781
2020-12-15 10:58:11 +01:00
Olivier Goffart
75f66ddaa1
Local discovery: always recurse within touched directory
If the file system watcher tells us a directory was modified, we should
recurse into it because it means it is probably a new directory

Issue #6804
2020-12-15 10:58:11 +01:00
Christian Kamm
e10e953c66
PropagateUpload: Avoid crash due to cascading aborts
https://sentry.io/owncloud/desktop-win-and-mac/issues/698694072/activity/
2020-12-15 10:58:11 +01:00
Christian Kamm
d1aedcfd3c
Discovery: restructure processFileAnalyzeLocalInfo 2020-12-15 10:58:10 +01:00
Christian Kamm
a2839bd40a
Discovery: easy conditions first in processFileAnalyzeRemoteInfo
Removing two levels of indent that way
2020-12-15 10:58:10 +01:00
Christian Kamm
76341904e9
Discovery: Add comments 2020-12-15 10:58:10 +01:00
Olivier Goffart
5a1c93d0ac
Discovery: make sure finished is not called twice, even in case of errors 2020-12-15 10:58:10 +01:00
Olivier Goffart
b86e1efc9a
Remove the backup deteciton code which was used for server < 9.1 2020-12-15 10:58:10 +01:00
Olivier Goffart
f666511a4b
Discovery: Remove stale DB entries
And test the Remove/Remove case.

This means we need to always query the database for all the entries.
This showed another small bug in the test in which sync item for virtual
files at the root could have a slash in front of them.
2020-12-15 10:58:10 +01:00
Christian Kamm
5683278fab
Discovery: Comments and visibility adjustments 2020-12-15 10:58:09 +01:00
Olivier Goffart
aa18e10ff5
Discovery: cleanups and comments 2020-12-15 10:58:09 +01:00
Olivier Goffart
b10b3e5eeb
Discovery: move checkMovePermissions to its own function 2020-12-15 10:58:09 +01:00
Olivier Goffart
edd866b32b
Discovery: Adjust the instruction in case of resolved conflict
When resolving a conflict because the file was just updated on the server,
we write all the metadata on the database immediatly, so INSTRUCITON_NONE
is enough and UPDATE_METADATA is not needed
2020-12-15 10:58:09 +01:00
Olivier Goffart
7cddaf82ab
ProcessDirectoryJob: always set _childModified to true, regardless the direction
This was like that to handle the case of CSYNC_INSTRUCTION_TYPE_CHANGE, but just add
a condition in this location.
2020-12-15 10:58:09 +01:00
Olivier Goffart
46510c2f39
Discovery phase: refactor some code in DiscoveryPhase::findAndCancelDeletedJob
Less code duplication
2020-12-15 10:58:09 +01:00
Olivier Goffart
1c2a3279bb
New Discovery Algorithm: more cleanups
- rename progress() to be more explicit
 - Make some more member of the discovery phase private
2020-12-15 10:58:09 +01:00
Olivier Goffart
2f8c77c54f
Fixup commit 835c9163374f42003aa2f7795ade3f4ff62c8877
The previous code considered the also HTTP 207 code without the
application/xml header to have this message.
httpCode 0 does not make much sense anyway.

This change the behavior to consider any 2xx without the xml header
to show this error message
2020-12-15 10:58:08 +01:00
Olivier Goffart
c009dae1ce
New discovery algorithm: fixups
Adapt reviews from ckamm in https://github.com/owncloud/client/pull/6738#pullrequestreview-164623532

- SyncJournalFileRecord: initialize everything inline
- Add more comments
- And some ENFORCE
2020-12-15 10:58:08 +01:00
Olivier Goffart
059f722b3b
Move Result to its own header 2020-12-15 10:58:08 +01:00
Olivier Goffart
afed46afcc
New discovery algorithm: Parallel PROPFIND 2020-12-15 10:58:08 +01:00
Olivier Goffart
c8eff3da2d
New Discovery algorithm: Remove the sync cleanup phase
Since we do not recurse within some directories, many files are not seen.

The stale entry will cleanup by themself as the sync engine try to remove
the files that are already removed.
Should we need to actually do this cleanup, it should be dotected in the
discovery.
2020-12-15 10:58:08 +01:00
Olivier Goffart
35c0cf4e59
New discovery algorithm: Set the originalFile for ignored files
This is used to show the name in the UI
2020-12-15 10:58:08 +01:00
Olivier Goffart
52dcfcb166
New Propagation algorithm: Fetch, and emit, the root etag
Remove the feature to concatenate etags as servers that don't
have a root etag are no longer suported
2020-12-15 10:58:07 +01:00
Olivier Goffart
d25d87e92c
New Discovery Algorithm: Ge tthe size of new folders
Also add a test that this works properly
2020-12-15 10:58:07 +01:00
Olivier Goffart
ef542ac83d
New Discovery algorithm: Split the process function even further
Move the finialization in its own function.
This allow to save a bit of code duplication.

Also change the order of the parameter in the constructor for consistency
2020-12-15 10:58:07 +01:00
Olivier Goffart
fc69dda246
New Discovery Algo: Refactor by splitting the processFile in two 2020-12-15 10:58:07 +01:00
Olivier Goffart
b7d827d6d0
New Discovery Algorithm: Handle rename of virtual files 2020-12-15 10:58:07 +01:00
Christian Kamm
e58a25d7c5
Virtual files: Renames propagate #6718 2020-12-15 10:58:07 +01:00
Olivier Goffart
3289675b8d
Folder Wizard: Disable the 'choose what to sync' treeview if virtual files are selected
Don't allow to configure virtual files with a selective sync blacklist
2020-12-15 10:58:07 +01:00
Olivier Goffart
d8fa8e270f
AccountSetting: Fix Small Memoy leak 2020-12-15 10:58:06 +01:00
Olivier Goffart
0ba9dc7dd1
AccountSettings: Force sync should wipe the blacklist
Issue #6757
2020-12-15 10:58:06 +01:00
Olivier Goffart
68ace415d4
Virtual files: don't check for new big folders
https://github.com/owncloud/client/issues/6724#issuecomment-417368475
2020-12-15 10:58:06 +01:00
Olivier Goffart
75a57778d9
Virtual files: Don't show selective sync
Issue #6724
2020-12-15 10:58:06 +01:00
Olivier Goffart
46aa8fd1f8
Socket API: add an option to replace existing files with virtual files
Issue #6726
2020-12-15 10:58:06 +01:00
Olivier Goffart
afc953b649
Update the minimum supported version
We want to warn if the server version is not supported and did not get
appropriate QA for this client version.

https://github.com/owncloud/enterprise/issues/2687
2020-12-15 10:58:05 +01:00
Markus Goetz
811ac50c05
CMakeLists.txt: Log the Qt version in compile 2020-12-15 10:58:05 +01:00
Christian Kamm
6b571b609c
Conflicts: Rename BasePath to InitialBasePath #6709 2020-12-15 10:58:05 +01:00
Olivier Goffart
d81ccbb0c1
Checksum: Add support for SHA256 and SHA3
In case, some day, the server also supports it
2020-12-15 10:58:05 +01:00
Olivier Goffart
bf6e4174c8
Move the checksum related routine from FileSystem to checksum, where they belong 2020-12-15 10:58:05 +01:00
Olivier Goffart
198b65e9fc
Fix a compilation error on windows 2020-12-15 10:58:05 +01:00
Olivier Goffart
21fe54fb13
New Discovery Algo: readability improvements
As proposed by ckamm on #6738
2020-12-15 10:58:04 +01:00
Olivier Goffart
3a06a7978f
New discovery algo: data finger print 2020-12-15 10:58:04 +01:00
Olivier Goffart
5683f1c33d
New Disco Algo: Handle spurious virtual files
Port commit 0b9049e6ff
for issue #6718
2020-12-15 10:58:04 +01:00
Olivier Goffart
c3f745fa76
Conflict handling: add the OC-ConflictBasePath header
Issue #6702
2020-12-15 10:58:04 +01:00
Olivier Goffart
358aadfb98
Account Settings: Add a text for folder using vierual files
Issue #6723
2020-12-15 10:58:04 +01:00
Olivier Goffart
73d933115a
Virtual Files: Set the mtime of the virtual file to that of the server
Note: When the server mtime is modified, the mtime of the file is not updated

Issue #6727
2020-12-15 10:58:04 +01:00
Olivier Goffart
a17a2a9c06
Virtual Files Mime Type: use "vnd." prefix instead of "x-"
As reported in https://github.com/owncloud/client/issues/6717#issuecomment-413703567
2020-12-15 10:58:03 +01:00
Christian Kamm
c10f103fb8
Virtual files: Renaming to virtual doesn't delete data #6718
Unfortunately to do this, the local update phase must write to the
database, creating a new side-effect and order dependency (local update
must run before remote update).
2020-12-15 10:58:03 +01:00
Christian Kamm
be0fa72fcb
RemotePermissions: Fix empty vs null
Fixes two bugs that appeared since the introduction of the struct:
- when reading permissions from the journal, null ("") was read as
  empty-not-null
- when reading permissinos from the server, empty ("") was read as null

Addresses #4608
2020-12-15 10:58:03 +01:00
Olivier Goffart
4837bc8d60
FolderStatusModel: Do not abort applying selective sync if one folder has an error
Issue #6675
2020-12-15 10:58:03 +01:00
Olivier Goffart
732069aaa7
SyncEngine: Fix the "direction" of the "all file delted" message when the server is reset
Using the direction of the "first" item is not enough as it might be a directory
which is a UPDATE_META_DATA,  or there can have been a few changes locally as well.

As reported on https://github.com/owncloud/client/issues/6317#issuecomment-412163113
2020-12-15 10:58:02 +01:00
Christian Kamm
f6ca649308
Do not require server replies to contain an mtime
The check was added for #6317 in commit
13eb64584f.

We did see missing mtimes in replies in tests with live servers though.
Possibly those were old incomplete responses cached in the stat cache?
2020-12-15 10:58:02 +01:00
Olivier Goffart
9401273daf
New discovery algorithm: Call FolderDiscovered 2020-12-15 10:58:02 +01:00
Olivier Goffart
a36ed56f01
New Discovery algorithm: Refactor a bit the way the signal are emited 2020-12-15 10:58:02 +01:00
Olivier Goffart
8f3ce6e520
Fix TestDownload: _ignore_hidden_files was not properly initialized 2020-12-15 10:58:02 +01:00
Olivier Goffart
b6487181c6
New Discovery Phase: fix a few FIXME
Some FIXME were already fixed.
2020-12-15 10:58:02 +01:00
Olivier Goffart
9863500ec1
New discovery algorithm: handle symlinks 2020-12-15 10:58:01 +01:00
Olivier Goffart
a9ec521bf1
New discovery algo: Handle Database error 2020-12-15 10:58:01 +01:00
Olivier Goffart
1bcbcd407c
New discovery phase: read the direct download URL and Cookie 2020-12-15 10:58:01 +01:00
Olivier Goffart
ca32eb1cf7
More cleanup of csync remains 2020-12-15 10:58:01 +01:00
Olivier Goffart
ac24cdbde6
New Discovery Algo: Permsission check 2020-12-15 10:58:01 +01:00
Olivier Goffart
0a6d2d0f46
Remove csync_update and csync_reconcile
I guess some other csync utilities can also be remove dnow but
that will be for later
2020-12-15 10:58:00 +01:00
Olivier Goffart
57068b0fd9
New Discovery Algorithm: handle server errors 2020-12-15 10:58:00 +01:00
Olivier Goffart
4031fb6d5b
Backport 64014dd374 2020-12-15 10:58:00 +01:00
Olivier Goffart
75a1f2d089
Convert p7.pl to a C++ test
This is just a translation of test/scripts/txpl/t7.pl to C++ using the test
framework.
2020-12-15 10:58:00 +01:00
Olivier Goffart
f4a5167812
New Discovery Algo: Support the DatabaseAndFilesystem mode for local discovery 2020-12-15 10:57:59 +01:00
Olivier Goffart
370923791d
New Disco algortihm: Fix TestChunkingNG::connectionDroppedBeforeEtagRecieved
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-12-15 10:57:59 +01:00
Olivier Goffart
710934bdbd
New disco algo: Fix TestSyncFileStatusTracker 2020-12-15 10:57:59 +01:00
Olivier Goffart
f926248937
New discovery algorithm: More work on virtual files 2020-12-15 10:57:59 +01:00
Olivier Goffart
35e40b58ca
New Disco algorithm: Type change (file to dir) 2020-12-15 10:57:59 +01:00
Olivier Goffart
1e8c37d3d6
New discovery algorithm: Virtual files
The commented tests lines were implementation details
2020-12-15 10:57:59 +01:00
Olivier Goffart
22d989e272
New discovery algorithm: Fix directory deletion
TestAllFilesDeleted passes
2020-12-15 10:57:59 +01:00
Olivier Goffart
d54e00488a
New Discovery algorithm
Some error handling. In particular for the case where there is a conflict
between files and directories.

SyncEngineTest and SyncMoveTest passes
2020-12-15 10:57:58 +01:00
Olivier Goffart
a384a2d1cb
New Discovery algorithm: Handle of move within a moved directory 2020-12-15 10:57:58 +01:00
Olivier Goffart
bdd1e72dda
New discovery algoritmh: more on Renames 2020-12-15 10:57:58 +01:00
Olivier Goffart
f43d07dc05
New Discovery algorithm: Check that the original file is still on the server while renaming 2020-12-15 10:57:58 +01:00
Olivier Goffart
7e36cc3fcb
New disco algorithm: Fix some moving
Fix TestSyncMove::testSelectiveSyncMovedFolder
2020-12-15 10:57:58 +01:00
Olivier Goffart
5a57a36729
New discovery algorithm: Local rename 2020-12-15 10:57:58 +01:00
Olivier Goffart
f9a9be59e6
New discovery algo: Remote move
TestSyncMove::testRemoteChangeInMovedFolder
2020-12-15 10:57:57 +01:00
Olivier Goffart
92ddc60900
Handle Encoding Problems
TestSyncEngine now passes
2020-12-15 10:57:57 +01:00
Olivier Goffart
501da58b10
Ignore Hidden Files 2020-12-15 10:57:57 +01:00
Olivier Goffart
8bf69cf0cd
Port the invalid napme regexp
TestSyncEngine::testInvalidFilenameRegex
2020-12-15 10:57:57 +01:00
Olivier Goffart
9efd21d70b
More progress
TestSyncEngine::testFakeConflict / TestSyncEngine::testSyncFileItemProperties
2020-12-15 10:57:57 +01:00
Olivier Goffart
e934f6b27b
New discovery algo
Make TestSyncEngine::testSelectiveSyncBug pass
2020-12-15 10:57:57 +01:00
Olivier Goffart
4066c1a004
New discovery: TestSyncEngine::testEmlLocalChecksum 2020-12-15 10:57:57 +01:00
Olivier Goffart
9bf417d930
New discovery algorithm
Some progress:
TestSyncEngine::testDirUpload
2020-12-15 10:57:56 +01:00
Olivier Goffart
ec681ab2a5
New discovery algorithm: Initial work.
SyncEngineTest testFileDownload is passing
2020-12-15 10:57:56 +01:00
Kevin Ottens
ba0d465e47
Revert "Merge pull request #1454 from nextcloud/syncjournal-del-prio"
This reverts commit d9fd9cfef2, reversing
changes made to 2dcf594fc6.
2020-12-15 10:57:56 +01:00
Dominik Schmidt
07f331717f
Rename ocsync library to ${APPLICATION_EXECUTABLE}_csync 2020-12-15 10:57:56 +01:00
Dominik Schmidt
aad928a6be
Install libocsync to lib/ without subfolder.
Installing to lib/${APPLICATION_EXECUTABLE} has caused a bunch of
irritations in the past and subtle annoying to fix bugs. To avoid name
clashes with branded clients ${APPLICATION_EXECUTABLE} becomes now
part of the filename instead of the subfolder.

The concrete motivation to change this now is that on Windows there
is no RPATH and it's not possible to run owncloud directly from the
Craft Root folder, which is nice when you're developing on Windows.

It would have been possible to change this just for Windows but as
written earlier this has caused lots of issues and thus I think it's
a good idea to just stay consistent accross platforms when touching it.
2020-12-15 10:57:56 +01:00
Olivier Goffart
0155a4fa73
Wizard + OAuth: Make opening a new browser after clicking back works again
Issue #6574

When there is an error in the advanced page, OwncloudAdvancedSetupPage::updateStatus
(and others) call completeChanged(), which is connected to
QWizardPrivate::_q_updateButtonStates which will re-enable the back button from the
last page.

When the user click "back" and re-open the browser, the account's credentials
already have a oauth token set. So the call to the API to get a new token fails
because we use the previous token instead of using the client's secret_id.
Fix this with the HttpCredentials::DontAddCredentialsAttribute.

Now, this is still not working because the session cookies are confusing the
server.  So we'll clear the cookies when re-opening the browser
2020-12-15 10:57:55 +01:00
Olivier Goffart
0d21936e95
FolderStatusModel: Fix crash when there is an error while expanding folders
In FolderStatusModel::slotLscolFinishedWithError, the call to parentInfo->resetSubs
deleted the 'job' and the reply 'r' which we accessed later to get the error code.

Fix this problem twice by
 1) Get the error code before caling resetSubs
 2) in FolderStatusModel::SubFolderInfo::resetSubs, call deleteLater instead of delete

Regression introduced in commit d69936e0

Issue #6562
2020-12-15 10:57:55 +01:00
Olivier Goffart
cf3846c565
csync: refactor csync_s::error_string to avoid valgrind error
The problem here is that we were sometimes allocating the error_string with
qstrdup, which need to be released with delete[] and not free().

Simplify the code by using QString instead.

```
==7230== Mismatched free() / delete / delete []
==7230==    at 0x4C2E10B: free (vg_replace_malloc.c:530)
==7230==    by 0x57C2321: csync_s::reinitialize() (csync.cpp:247)
==7230==    by 0x548130F: OCC::SyncEngine::finalize(bool) (syncengine.cpp:1212)
==7230==    by 0x5481223: OCC::SyncEngine::handleSyncError(csync_s*, char const*) (syncengine.cpp:746)
==7230==    by 0x5483E78: OCC::SyncEngine::slotDiscoveryJobFinished(int) (syncengine.cpp:965)
==7230==    by 0x5495E34: QtPrivate::FunctorCall<QtPrivate::IndexesList<0>, QtPrivate::List<int>, void, void (OCC::SyncEngine::*)(int)>::call(void (OCC::SyncEngine::*)(int), OCC::SyncEngine*, void**) (qobjectdefs_impl.h:134)
==7230==    by 0x5495D92: void QtPrivate::FunctionPointer<void (OCC::SyncEngine::*)(int)>::call<QtPrivate::List<int>, void>(void (OCC::SyncEngine::*)(int), OCC::SyncEngine*, void**) (qobjectdefs_impl.h:167)
==7230==    by 0x5495CB5: QtPrivate::QSlotObject<void (OCC::SyncEngine::*)(int), QtPrivate::List<int>, void>::impl(int, QtPrivate::QSlotObjectBase*, QObject*, void**, bool*) (qobjectdefs_impl.h:396)
==7230==    by 0xA9BF2E1: QObject::event(QEvent*) (in /usr/lib/libQt5Core.so.5.11.0)
==7230==    by 0x64BE983: QApplicationPrivate::notify_helper(QObject*, QEvent*) (in /usr/lib/libQt5Widgets.so.5.11.0)
==7230==    by 0x64C625A: QApplication::notify(QObject*, QEvent*) (in /usr/lib/libQt5Widgets.so.5.11.0)
==7230==    by 0xA994BC8: QCoreApplication::notifyInternal2(QObject*, QEvent*) (in /usr/lib/libQt5Core.so.5.11.0)
==7230==  Address 0x225b2640 is 0 bytes inside a block of size 50 alloc'd
==7230==    at 0x4C2DC6F: operator new[](unsigned long) (vg_replace_malloc.c:423)
==7230==    by 0xA7E8FC8: qstrdup(char const*) (in /usr/lib/libQt5Core.so.5.11.0)
==7230==    by 0x53F5750: OCC::DiscoveryJob::remote_vio_opendir_hook(char const*, void*) (discoveryphase.cpp:666)
==7230==    by 0x57E1278: csync_vio_opendir(csync_s*, char const*) (csync_vio.cpp:39)
==7230==    by 0x57D718F: csync_ftw(csync_s*, char const*, int (*)(csync_s*, std::unique_ptr<csync_file_stat_s, std::default_delete<csync_file_stat_s> >), unsigned int) (csync_update.cpp:674)
==7230==    by 0x57C1B05: csync_update(csync_s*) (csync.cpp:109)
==7230==    by 0x53F5BCC: OCC::DiscoveryJob::start() (discoveryphase.cpp:718)
==7230==    by 0x54B8F74: OCC::DiscoveryJob::qt_static_metacall(QObject*, QMetaObject::Call, int, void**) (moc_discoveryphase.cpp:494)
==7230==    by 0xA9BF2E1: QObject::event(QEvent*) (in /usr/lib/libQt5Core.so.5.11.0)
==7230==    by 0x64BE983: QApplicationPrivate::notify_helper(QObject*, QEvent*) (in /usr/lib/libQt5Widgets.so.5.11.0)
==7230==    by 0x64C625A: QApplication::notify(QObject*, QEvent*) (in /usr/lib/libQt5Widgets.so.5.11.0)
==7230==    by 0xA994BC8: QCoreApplication::notifyInternal2(QObject*, QEvent*) (in /usr/lib/libQt5Core.so.5.11.0)
==7230==
```
2020-12-15 10:57:55 +01:00
Olivier Goffart
857afd32df
PropagateDownload: Don't discard the body of error message
We want to keep the body so we can get the message from it
(Issue #6459)

TestDownload::testErrorMessage did not fail because the FakeErrorReply
did not emit readyRead and did not implement bytesAvailable.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-12-15 10:57:55 +01:00
Olivier Goffart
4ee244190b
Virtual Files: Allow to download a folder recursively from the socket API
Issue: #6466
2020-12-15 10:57:54 +01:00
Christian Kamm
6dba2e8b06
Virtual files: Wipe virtual after download completes, not before
Otherwise a interrupted or unsuccessful download would mean that the
download-intend was forgotten. The next sync would reestablish the
virtual file instead.
2020-12-15 10:57:54 +01:00
Olivier Goffart
530614bf78
Rename SocketApi::FileData::accountRelativePath to serverRelativePath
As discussed in issue #6552
2020-12-15 10:57:54 +01:00
Olivier Goffart
ced5dfb8ee
Share dialog: use the original name and not the virtual file name
When sharing a virtual file, we should actually use the original file name
not the virtual file name

Issue: #6461
2020-12-15 10:57:54 +01:00
Christian Kamm
a6c19572a2
Virtual files: Only remove virtual file once on download
With thanks to @ogoffart for spotting the problem.
2020-12-15 10:57:54 +01:00
Christian Kamm
97f7b5abeb
Settings migration: Preserve future settings where possible
See discussion in #6506
2020-12-15 10:57:54 +01:00
Christian Kamm
87ba4e6b9c
Config: Add version flags to accounts and folders
Also, if there is too-new configuration, backup the file, show a
warning message asking the user whether it's ok to discard the
configuration from the future.

See #6504
2020-12-15 10:57:53 +01:00
Christian Kamm
aa6f5f59c4
Rename Placeholders to Virtual Files in code #6531 2020-12-15 10:57:53 +01:00
Olivier Goffart
a877a9d472
SyncRunFileLog: Add the requestid
https://github.com/owncloud/client/pull/6427#issuecomment-383879509
2020-12-15 10:57:53 +01:00
Olivier Goffart
09cacc4cd4
Blacklist: remember the X-Request-ID
Issue #6420
Store the X-Request-ID in the SyncFileItem and also in the blacklist.
Note that for consistency reason, the X-Request-ID is also in the
SyncFileItem if the request succeeds.

Currently there is no UI to access it, but it can be queried with sql
commands
2020-12-15 10:57:53 +01:00
Christian Kamm
7da4bb4c20
Wizard: Add a "(recommended)" to the main option #6470 2020-12-15 10:57:53 +01:00
Christian Kamm
ddeeecd061
Rename "placeholder" feature 2020-12-15 10:57:53 +01:00
Christian Kamm
4e3f2f755a
Placeholder: Stop adding ignore pattern
Because we can't make older clients preserve the version directive that
was attached to it.

See #6504 and #6498
2020-12-15 10:57:52 +01:00
Christian Kamm
8b6ac63ddb
Placeholder: Don't contain "stub" 2020-12-15 10:57:52 +01:00
Christian Kamm
27b65cbc69
Placeholders: Save to key that's invisible to <2.5 clients #6504 2020-12-15 10:57:52 +01:00
Olivier Goffart
3272f3b72b
FolderWizard: fix crash when experimental feature are disabled
The _placeholderCheckBox only exists if the experimental features are enabled

Found via the crash reporter
https://sentry.io/owncloud/desktop-win-and-mac/issues/556407777/
2020-12-15 10:57:52 +01:00
Markus Goetz
215afba89e
Share Dialog: Disable workaround for macOS too #6185 2020-12-15 10:57:52 +01:00
Markus Goetz
8dcfd50a7d
Placeholders: Support for macOS #6290 2020-12-15 10:57:51 +01:00
Christian Kamm
765c12dae1
LocalDiscoveryTracker: Separate from Folder and move to libsync
To allow relevant code to be closer together and for testing in
unittests without having to get a gui Folder.

See #6120
2020-12-15 10:57:51 +01:00
Olivier Goffart
7969067f80
Logger: Guard zlib usage by ZLIB_FOUND
Similar to the use for the checksum.
I know that zlib is required in principle, but i don't have it
in one of my test building environment, and it is easier to just
disable it.
2020-12-15 10:57:51 +01:00
Kevin Ottens
b1dc14977f
Get TestDownload to pass
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-12-15 10:57:51 +01:00
Olivier Goffart
4eb8997519
Download: Remove useless code and add a test
From issue #7015, the code is wrong because the path is the file system path and
not the path on the DB.
But since this is a conflict, this means the reconcile will still want to download
the file from the server next sync, so we need not to worry about this case
2020-12-15 10:57:50 +01:00