I went full steam on making it all public which is not really required,
it's only really QtWebSockets we're after. Could always be fine tuned
later on if this works as expected.
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
This would only happen if the parent of the newly created folder would
be explicitly set to online only, hence why it went under the radar
previously.
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
This account object was really only used during the initialization phase
or for forgetting the sensitive data. So let's receive it as parameter
there and pass it on from job to job as needed.
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
The OS might request the same file again if we take too much time to
fulfill a request. So in case it's queueing the same one again instead
of bailing out just fail the second one and let the first one finish
properly.
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
In case we'd been to slow to fullfill or we're still processing a
cancelled request better not just crash. We still log the issue though.
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
This got broken previously when trying to reconcile during a
cherry-pick. This also repairs the case when the theme tells us the
default is to not select anything (not that we're using this right now
but in case we'd like to brand this).
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
We now favor APPLICATION_ICON_SET to isBranded() regarding the decision
to use PNG or SVG for icons.
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
Indeed, that file size will almost always change between the 1 byte
placeholder and the hydrated file. Only when using the CfAPI on Windows
this won't be the case since because it will expose the original size
even for placeholders.
Also worth noting: the suffix backend didn't hit that case since the
filename changes (with suffix for placeholders, without for hydrated
files).
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
Ideally this will end up being the backend we use for both Linux and
macOS but that will require work with desktop environments on the Linux
side and to reverse engineering at least on xattr value on macOS.
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
There's been a confusion between the chunk number and the chunk
offset leading to corruptions... Let's pass the proper offset to
the UploadDevice again.
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
MSVC having so useless error messages it didn't quite point to the root
cause of the issue... it turns out that through the maze of macros
defined in the windows API, there's one which impacted the function
pointer definition of CfCloseHandle which would then not convert to
FileHandle::Deleter as expected. So I end up wrapping it in a lambda to
help... luckily this kind of lambdas decay into a simple function
pointer so there's likely no overhead it's just to coerce the compiler
into doing the right thing.
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
For some reason MSVC manages to deduce the right constructor in Win64
mode but not in Win32 mode. So let's be more explicit about what we
return.
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
Otherwise this would attempt to download the file everytime we check the
direct editor for a given file which wouldn't be adequate. Would also
lead to a deadlock in our case since that would happen in the main
thread and implicit hydration is driven from there as well.
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
This comes with a test simulating an open request coming from another
process (although in our case it's really just a thread). The actual
hydration works as expected by cfapi, handling of encrypted files is for
now missing.
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
For now this implements only the logic necessary to get a test suite
equivalent to the TestSyncVirtualFiles one to pass. It doesn't (yet)
honor request to fetch files from the system.
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
Shouldn't be translated.
Just checked https://en.wikipedia.org/wiki/HTTP_ETag and all available localized pages.
Signed-off-by: rakekniven <mark.ziegler@rakekniven.de>
We will have all the code in public anyway so it can just be compiled
in. Thus no need to go through the plugin loading dance. Replaced the
loading with factory functions. Kept mostly the same structure
otherwise.
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
It felt odd to be able to control the encryption status in the settings
dialog but not the availability. Also availability was controllable on
the root, so let's make it widely available.
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
This got removed from the settings since in that case selective sync
isn't supported. Unfortunately that's also necessary to display them to
allow encrypting folders via the context menu.
So we display the subfolders again but in the case of VFS we disable the
ability to (un)check them. They just have an icon, a name and a context
menu.
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
Otherwise we would not display the E2EE message if the account was
connected before the creation of the settings dialog. This was likely
caused by the delayed creation of the settings dialog.
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
If we use those encrypted propagation code paths, we already know from
the discovery phase (and thus the journal db) that the folders are
encrypted so no need to check again.
This will remove another expensive round trip with the server.
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
This step isn't necessary anymore, no one looks at ClientSideEncryption
for that information anyway.
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
We used the same approach than for the FolderStatusModel by getting the
is-encrypted property straight from the LSCOL job.
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
Thanks to the new discovery algorithm, we got all the freshest E2EE
information straight from the database so reuse it instead of going
through an in memory copy.
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
It had a different path convention than all the other jobs, most likely
for legacy reasons because of the tight coupling it had to the settings
dialog.
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
No need to look for a token on the outside we can just work properly by
keeping all the state encapsulated in the job.
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
Thanks to the new discovery algorithm we got both mangled and original
file names in the item so no need to go through the database for
nothing.
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
Now we pull the encrypted metadata during the discovery which is a
better approach than before. This shall remove the need for some of the
deep propfinds we have been using so far. It already simplifies the code
a bit for the download scenario.
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
We now enforce the use of QStringLiteral and friends in some places,
but that feel through the cracks for some of the Windows specific code.
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
We also don't have a version entry in the db if thne db is new.
As I don't expect regular updates from 1.5 this message can just be removed.
Fixes: #8004
[ fatal default ]:ASSERT: "last < rowCount(parent)" in file C:\_\17a9f6ae\qtbase-everywhere-src-5.12.9\src\corelib\itemmodels\qabstractitemmodel.cpp, line 2787
e1ca612c5d3087e02f6b7f8e454224e0b88e82ad stopped adding sub folders so we can return the actual size.
The OAuth authentication brings the broweser to the front, once thats done the wizard continues.
But the wizard ist now most probably hidden behind the browser
This code was actually not breaking most cookie handling by accident.
As the raw cookies where not split properly we added cookies with values like
"key: val; key2 = val2; key3 = val3"
When the code was corrected we overwrote the newer values in the jar with
the old ones from a request.
This was done because the propagator jobs where running in a thread a long
time ago, but this is no longer the case.
(Also QAtomicInt::load is marked as deprecated now)
For a usual file sync event we check for actual changes in the local file,
after an unlock the local file might be unchanged so we need to sync it anyhow.
Fixes: owncloud/enterprise#3609
The original code from csync was stopping at any error.
But we have been whitelisting soeme http error code one by one
to ignore the directory instead of aborting the sync.
However, as there are more requests to continue the sync in case
of error, just ignore most HTTP errors
Issue #7586
On windows we do a test to know if we should change the case of the files,
but that conflict with the test that checks if the file was still there
when the filename is actually the same. Which can happen with virtual files
as they have two representation (the one with and without suffix).
When an item is downloaded because it is restored, it shall be shown in the
sync protocol.
(It is also going to be shown in the not synchronized for a short while, but
that's fine)
When the gui thread blocks for several seconds it's possible for the
ConnectionValidator to timeout and decide that the account is
unreachable. It will then terminate all sync runs.
Increasing the timeout makes this less likely to happen. The tradeoff is
that real disconnects will not be detected as quickly.
This does not address the root cause but makes the symptom less likely
to appear.
Previously the source was deleted (or attempted to be deleted), even if
the new location was not acceptable for upload. This could make data
unavilable on the server.
For #7410
By introducing a PropagateRootDirectory job that explicitly
separates the directory deletion jobs from all the other jobs.
Note that this means that if there are errors in subJobs the
dirDeletionJobs won't get executed.
Previously the job would only become "active" when the downloads
started. That meant that arbitrarily many hash computations could be
queued at the same time.
Since the the file was opened during future creation this could lead to
a "too many open files" problem if there were lots of new-new conflicts.
To change this:
- Make PropagateDownload become active when computing a hash
asynchronously.
- Make the computation future open the file only once it gets run. This
will make it less likely for this problem to occur even if thousands
of these futures are queued.
For #7372
Previously fatal error texts were duplicated: Once they entered the
SyncResult via the SyncFileItem and once via syncError().
syncError is intended for folder-wide sync issues that are not pinned
to particular files. Thus that duplicated path is removed.
For #5088
Previously the pin states of deleted files stayed in the 'flags'
database and could be inadvertently reused when a new file with the same
name appeared. Now they are deleted.
To make this work right, the meaning of the 'path' column in the 'flags'
table was changed: Previously it never had the .owncloud file suffix.
Now it's the same as in metadata.path.
This takes the safe parts from #7274 for inclusion in 2.6. The more
elaborate database schema changes (why use 'path' the join the two
tables in the first place?) shall go into master.
Previously RequestEtagJob did return the etag verbatim (including extra
quotes) while the db had the parsed form. That caused the etag
comparison during discovery move detection to always fail. The test
didn't catch it because the etags there didn't have quotes.
Now:
- RequestEtagJob will parse the etag, leading to a consistent format
- Tests have etags with quotes, detecting the problem
- Close the UploadDevice to close the QFile after the PUT job is done.
This allows winvfs to get an oplock on the file later.
- Don't rely on QFile::fileName() to be valid after
openAndSeekFileSharedRead() was called. The way it is openend on
Windows makes it have an empty filename.
If one adds a new file to an online-only folder the previous behavior
was to upload the file in one sync and dehydrate it in the next. Now
these new files get set to Unspecified pin state, making them retain
their data.
Instead of all at once, to reduce peak memory use.
Changing UploadDevice in this way requires keeping the file open for the
duration of the upload. It also means changes to open(), seek(), close()
to ensure that uses of the device work right when a request needs to
be resent.
Since Qt does not yet transparently resend HTTP2 requests in some cases
we do it manually.
The test showed a problem where the initial non-200 reply would close
the target temporary file and the follow-up request couldn't store any
data. Removing that close() call is safe because there also is a
_saveBodyToFile flag that guards writes to the target file.
The previous patch ensured that the sqlite temporaries weren't deleted
and recreated for every sync run, but there was still time between
client startup and the first sync run where they would have the
"needs-sync" icon.
Previously "no-availability" meant db-error and querying the
availability of a nonexistant path returned AllHydrated.
Now, the availability has a DbError and a NoSuchItem error case.
Saying "Currently available locally" sounds more like an indicator than
"Availably locally" does. Centralizing translations avoids consistency
issues between shell context menus and sync folder context menu.
The db-close operation is likely a leftover from when the SyncEngine
owned its own db connection and serves no purpose anymore.
Closing the db causes the removal of the temporary wal and shm files.
These files are recreated when the db is opened again, which happens
almost immediately.
This is a problem for winvfs because the delete-recreate step wipes the
exclusion state on these files just after the sync is done. That meant
that the db temporaries permanently had a "needs sync" icon marker shown
in the explorer.
Avoiding reopening the db also reduces the number of log messages per
sync.
Previously these result codes during remote discovery of the sync root
would not cause an error and the discovery would get stuck.
Also extends RemoteDiscovery tests to check for errors on the root item.
Previously if one set the instruction to ERROR while forgetting to set
an error status, it'd propagate as FileIgnored. Now the default is
NormalError for INSTRUCTION_ERROR and FileIgnored for
INSTRUCTION_IGNORE.
The idea is that the user's question is "is this folder's data available
offline?" and not "does this folder have AlwaysLocal pin state?".
The the answers to the two questions can differ: an always-local
folder can have subitems that are not always-local and are dehydrated.
The new availability enum intends to describe the answer to the user's
actual question and can be derived from pin states. If pin states aren't
stored in the database the way of calculating availability will depend
on the vfs plugin.
The pin state is a per-item attribute that has an effect on _type:
AlwaysLocal dehydrated files will be marked for hydration and OnlineOnly
hydrated files will be marked for dehydration.
Where exactly this effect materializes depends on how the pin states are
stored. If they're stored in the db (suffix) the dbEntry._type is
changed during the discovery.
If the pin state is stored in the filesystem, the localEntry._type must
be adjusted by the plugin's stat callback.
This patch makes pin states behave more consistently between plugins.
Previously with suffix-vfs pin states only had an effect on new remote
files. Now the effect of pinning or unpinning files or directories is as
documented and similar to other plugins.
1. The _firstJob is usually deleted by the time the PropagateDirectory
finishes. (deleteLater() is called early)
2. The PropagateDirectory::_item and PropagateRemoteMkdir::_item point
to the same SyncFileItem anyway. This code is a leftover from when
each job had its own instance.
Previously removing the vfs suffix of a file always triggered a
conflict. Now it may just cause a file download.
This was done because users expected symmetry in the rename actions and
renaming foo -> foo.owncloud already triggers the "make the file
virtual" action. Now foo.owncloud -> foo triggers the "download the
contents" action.
Kevin Ottens