Previously this wasn't happening for errors that were not
NormalErrors because they don't end up in the blacklist.
This revises the resetting logic to be independent of the
error blacklist and make use of UploadInfo::errorCount
instead.
412 errors should reset chunked uploads because they might be
indicative of a checksum error.
Additionally, server bugs might require that additional
errors cause an upload reset. To allow that, a new capability
is added that can be used to advise the client about this.
The rules to select the webdav url are now:
- If the server reports that the new chunking algorithm is working,
always use remote.php/dav/files/<username>
This capability can be overriden with an environment variable
- Otherwise, use the dav path provided by the theme, which defaults to
remote.php/webdav
This means that with the newer server, the branding can no longer override
the webdav URL. If there is still an usecase for the branding to do so, we
need to find another way to override it. But it is now more complicated to
configure as might need include the username and need different endpoint
depending on the operations (chunks or not)
Issue #4007
We are going to change the webdav path depending on the capabilities.
But the SyncEngine and csync might have been created before the capabilities
are retrieved.
The main raison why we gave the path to the sync engine was to pass it to csync.
But the thing is that csync don't need anymore this url as everything is done by the
discovery classes in libsync that use the network jobs that use the account for the urls.
So csync do not need the remote URI.
shortenFilename in folderstatusmodel.cpp was useless because the string is the
_file of a SyncFileItem which is the relative file name, that name never
starts with owncloud://.
All the csync test creates the folder because csync use to check if the folder
exists. But we don't need to do that anymore
Two bugs:
- The change filed are not considered as move, they are re-downloaded
but the old file was not removed from the database. The change in
owncloudpropagator.cpp takes care of removing the old entries.
- Next sync would then remove the file in the server in the old folder
This was not a problem until we start reusing the sync engine, and
that the _renamedFolders map is not cleared. We were before deleting
a non-existing file. But now we delete the actual file.
Also improve the tests to be able to do move on the server.
This include support for file id.
Issue #5192
In case of the root directory, it may happen that the _item
is empty and the _item->_status is NoStatus. But we still need to report
the proper success or error of the whole propagation. We should really
use _hasError for that. However, _hasError is also defined to NoStatus
if there was no error, so in that case we need to set Success.
This fixes the problem in which the data-fingerprint is not saved on the
database because the SyncEngine think that the sync failed. (Issue #5185)
This would happen if the directory would first need to be created
through an mkdir propagation job. This job's itemCompleted signal
would trigger the directory to show as SYNC even though its children
are still propagating.
Fix the issue by tracking the sync count for each file, affecting
its parents. This allows us to get rid of the O(n) vector lookup
for each status query, and properly track the hierachical sync
status of a directory.
This also removes the itemCompleted signal emission from the
PropagateDirectory job. Since we only needed for overlay icons, and
since this job doesn't do any direct propagation, we can remove it
to ensure that we won't call itemCompleted twice for the item attached
to Propagate*Mkdir jobs (since the PropagateDirectory is backed by
the same SyncFileItem, instruction and status).
The current way of tracking the need to update the metadata without
propagation using a separate flag makes it difficult to track
priorities between the local and remote tree. The logic is also
difficult to logically cover since the possibilities matrix isn't
100% covered, leaving the flag only used in a few situations
(mostly involving folders, but not only).
The reason we need to change this is to be able to track the sync
state of files for overlay icons. The instruction alone can't be
used since CSYNC_INSTRUCTION_SYNC is used for folders even though
they won't be propagated. Removing this logic is however not possible
without using something else than CSYNC_INSTRUCTION_NONE since too
many codepath interpret (rightfully) this as meaning "nothing to do".
This patch adds a new CSYNC_INSTRUCTION_UPDATE_METADATA instruction
to let the update and reconcile steps tell the SyncEngine to update
the metadata of a file without any propagation. Other flags are left
to be interpretted by the implementation as implicitly needing
metadata update or not, as this was already the case for most file
propagation jobs. For example, CSYNC_INSTRUCTION_NEW for directories
now also implicitly update the metadata.
Since it's not impossible for folders to emit CSYNC_INSTRUCTION_SYNC
or CSYNC_INSTRUCTION_CONFLICT, the corresponding code paths in the
sync engine have been removed.
Since the reconcile step can now know if the local tree needs metadata
update while the remote side might want propagation, the
localMetadataUpdate logic in SyncEngine::treewalkFile now simply use
a CSYNC_INSTRUCTION_UPDATE_METADATA for the local side, which is now
implemented as a different database query.
Missing deleteLater when the CleanupPollsJob aborts.
This is only a problem if the SyncEngine is kept alive a long time. Which is
usually not the case in the configuration where poll jobs are used.
* Add checksums/supportedTypes and checksums/preferredUploadType
capabilities. The default is that no checksum types are supported.
* Remove the transmissionChecksum config option. Servers must now
use the capabilities to indicate that they are fine with the
client sending checksums.
Note: This intentionally breaks brandings that overrode
Theme::transmissionChecksum. The override must be removed and the
server's capabilities must be adjusted to include the new values.
Helps with small file sync #331
When I benchmarked this, it went up to 6 parallelism and
was about 1/3 faster than the previous fixed 3 parallelism.
Doing more than 6 is dangerous because QNAM limits to 6 TCP
connections and also the server might become a bottleneck.
Should also help for #4081
Added a new "chunkSize" entry in the General group of the owncloud.cfg
which can be set to the size, in bytes, of the chunks.
This allow user with hude bandwidth to select more optimal chunk size
Issue #4354
* Ensure every time a file becomes a directory or the other way around
the item is flagged as INSTRUCTION_TYPE_CHANGE.
* Delete the badly-typed entity if necessary in the propagation jobs.
The error status of children should only be used for the etag logic.
The SocketApi uses a path matching system to do this and the UI should
report errors only for individual involved files/directories.
Since the presence of any path in SyncEngine::_syncedItems
would translate in a SYNC status, platforms that don't refresh
all their status cache after an UPDATE_VIEW message like OS X
or Windows would keep displaying that status even after all
files are successfully synchronized.
- Read SyncFileItem::_status to determine the status to display mid-sync
- Match moved paths also to _renameTarget since this might be the
path to match
- Make sure that PropagateDirectory jobs also set SyncFileItem::_status
properly
Because that's what's going on. A job can 'complete an item' or 'finish'.
Note that several jobs could complete the same item: a new directory
will complete on the PropagateRemoteMkdir and the PropagateDirectory
jobs.
Previously, PropagateDirectory jobs didn't emit the completed() signal.
Now that they do, we need to make sure to not add extra lines to the
protocol widget for them.
To accomplish that, the jobCompleted() signal now also contains the job
that completed the item.
When user wants to limit the bandwidth, he does not care about speed
anymore. And parallelism on slow network might cause problems.
For issue #3382
Will also help for #3095
(cherry picked from commit b20f29f22797367c7aa92bd74389c99b10c852a4)
This fixes a few issues with the new FinderSync integration on OSX which
can't easily clear its status cache when receiving an UPDATE_VIEW message
except by unregistering the folder, but which causes flickering.
This fix should also make the regeneration of the cache unnecessary on other
platforms through possibly expensive RETRIEVE_FILE_STATUS commands.
Use a QSharedPointer to keep the same ownership and
continue passing the SyncFileItems as a const& when
ownership isn't taken. This allows sharing the same
allocations between the jobs and the result vectors.
This saves about 20MB of memory (off 120MB) once all
jobs are created.
The problem occurs because of the sorting of items when we have things like
bigfolder
bigfolder/bigsubfolder
bigfolder-2
Then, because dashes come before slash in ascii, the bigfolder-2 would come before its content
and the propagator would thinkg bigfolder is empty and save the etag before it is processed
Should fix issue #2832
* Use a shared pointer to Account everywhere to ensure
the instance stays alive long enough for a sync to terminate
* Folder is now tied to an AccountState
* SyncEngine and OwncloudPropagator tie to an Account and use that
for all jobs they run
Issue: Since the setup wizard currently always replaces the
account, it will always wipe all folder definitions, even when
the actual changes to the account were minor.
We need Qt 5.3.3.
If the compiled qt version is Qt 5 but smaller than Qt 5.3.3, do a runtime
check.
OWNCLOUD_NEW_BANDWIDTH_LIMITING is still there for a special setup in which
we patch Qt with the patches
Christian Kamm