1
0
Fork 0
Commit Graph

102 Commits

Author SHA1 Message Date
Jocelyn Turcotte b7553d5bdf Upgrade some qCDebug to qCInfo or qCWarning
Use qCInfo for anything that has general value for support and
development. Use qCWarning for any recoverable error and qCCritical
for anything that could result in data loss or would identify a serious
issue with the code.

Issue #5647
2017-05-11 17:22:59 +02:00
Jocelyn Turcotte 4ad190a558 Use Qt logging categories for logging
This gives more insight about the logs and allow setting fine-tuned
logging rules. The categories are set to only output Info by default
so this allows us to provide more concise logging while keeping the
ability to extract more information for a specific category when
developping or debugging customer issues.

Issue #5647
2017-05-11 17:22:59 +02:00
Kaustubh Welankar 70c3514996 Removed Magic Numbers in propagator #5685 (#5705) 2017-04-20 11:14:53 +02:00
Christian Kamm 35af03b2e5 Improve http error messages; cleanup
By default QNetworkReply::errorString() often produces messages like
   "Error downloading <url> - server replied: <reason>"
but the "downloading" part invariably confuses people since the
error might very well have been produced by a PUT request.

This commit produces clearer error messages for HTTP errors.

Additionally:
* Remove some unnecessary null checks from slots connected to
  network job signals and document that these signals never send
  null replies.
* There was a bug where AbstractNetworkJob::_timedout wasn't
  set when derived classes overrode slotTimeout. We now ensure
  it's always set by disallowing overrides of slotTimeout.
  Instead it now calls onTimedOut, which allows custom handling.
* Several subclasses declared errorString, isTimedOut. Move
  these to AbstractNetworkJob.
* Unify handling of OC-ErrorString (via the new, general
  Job::errorString)
* Add documentation in various places.
2017-04-04 09:27:37 +02:00
Christian Kamm e86499d990 dynamic chunking: cleanup, fixes, improvements
* make target duration a client option instead of a capability
* simplify algorithm for determining chunk size significantly
* preserve chunk size for the whole propagation, not just per upload
* move options to SyncOptions to avoid depending on ConfigFile
  in the propagator
* move chunk-size adjustment to after a chunk finishes, not when
  a new chunk starts
2017-03-28 11:32:10 +02:00
Piotr M 88e0f97798 dynamic chunking prototype 2017-03-27 13:43:39 +02:00
Christian Kamm 4a1a5fa076 AbstractNetworkJob: Improve redirect handling #5555
* For requests:
  - reuse the original QNetworkRequest, so headers and attributes
    are the same as in the original request
  - determine the original http method from the reply and the request
    attributes
  - keep the original request body around such that it can be sent
    again in case the request is redirected

* Simplify the interface that is used for creating new requests in
  AbstractNetworkJob.
2017-03-07 13:18:01 +01:00
Jocelyn Turcotte f6665ccc81 Pack boolean members in PropagateUploadFileCommon #5457 2017-01-26 17:13:54 +01:00
Jocelyn Turcotte f1e9be4fa8 Don't use the StopWatch for each job in production builds #5456
This accounted for more than 10% of the peak memory usage while only
providing value through the debugging log.
2017-01-26 17:13:54 +01:00
Jocelyn Turcotte 9db23d4df1 Remove SyncFileItem::_requestDuration #5456
This will save a QElapsedTimer in each job that was used only for
.owncloudsync.log (for which the request duration doesn't bring much
value).
2017-01-26 17:13:54 +01:00
Olivier Goffart d6fdda8efa ChunkingNG: add '0' padding on the filename (#5476)
The server sorts the chunk by name alphabetically. So if we want to keep
the chunk in order, we need to add a few '0' in front of the chunk name
2017-01-20 16:03:50 +01:00
Christian Kamm 29b39acfbe PropagatorJob: Access propagator through function #5453 2017-01-17 14:47:38 +01:00
ckamm d76e0ec6d8 Reset stuck chunked uploads eventually #5344 (#5443)
Previously this wasn't happening for errors that were not
NormalErrors because they don't end up in the blacklist.

This revises the resetting logic to be independent of the
error blacklist and make use of UploadInfo::errorCount
instead.

412 errors should reset chunked uploads because they might be
indicative of a checksum error.

Additionally, server bugs might require that additional
errors cause an upload reset. To allow that, a new capability
is added that can be used to advise the client about this.
2017-01-13 14:44:45 +01:00
Olivier Goffart 15f2b911d9 ChunkingNG: remove stale files when resuming 2016-11-04 16:43:01 +01:00
Olivier Goffart 0960058842 Merge remote-tracking branch 'origin/master' into chunking-ng 2016-10-20 09:25:03 +02:00
Markus Goetz 8c5ea8dc90 Propagator: Also upload more in parallel (#5230)
Fix for https://github.com/owncloud/client/issues/4986#issuecomment-227071801

This is before we have bundling at some point.
2016-10-14 14:23:55 +02:00
Olivier Goffart 7c75a39bc1 Chunking-NG: Some fixup after feedback from the pull request #5102 2016-09-10 12:30:14 +02:00
Olivier Goffart 818b5854ce Chunking-NG: Qt4 compile 2016-08-31 10:28:44 +02:00
Olivier Goffart fad387b6b8 Chunking-Ng: Resume 2016-08-31 10:28:15 +02:00
Olivier Goffart a1558100b8 WIP: new chunking algorithm
Current limitations of this WiP
 - No resuming implemented yet
 - No parallel chunks
 - Hackish way to get the webdav paths
2016-08-31 10:28:15 +02:00
Olivier Goffart 4f3f642da6 Upload: refactor the upload in two classes so the new chuning can be implemented 2016-08-31 10:28:15 +02:00
Olivier Goffart d2bde5489f Propagator, remove the QNAM in the name as the alternative has been removed a long time ago 2016-08-31 10:25:06 +02:00
Olivier Goffart d12c0939b9 Upload: Chunk size can be changed in the config file
Added a new "chunkSize" entry in the General group of the owncloud.cfg
which can be set to the size, in bytes, of the chunks.

This allow user with hude bandwidth to select more optimal chunk size

Issue #4354
2016-01-15 13:16:52 +01:00
Christian Kamm abf5a5ad1e Propagation: Fix dir <-> file changes propagating to server #4302
* Ensure every time a file becomes a directory or the other way around
  the item is flagged as INSTRUCTION_TYPE_CHANGE.
* Delete the badly-typed entity if necessary in the propagation jobs.
2016-01-08 05:38:08 +01:00
Christian Kamm a25f094c4c Checksums: Don't reupload if size and checksum are unchanged #3235
* Compute the content checksum (in addition to the optional
  transmission checksum) during upload (.eml files only)

* Add hook to compute and compare the checksum in csync_update

* Add content checksum to database, remove transmission checksum
2015-11-23 11:59:56 +01:00
Christian Kamm 0354289795 Checksums: Improvement in async computation and validation #3735 2015-10-28 09:56:39 +01:00
Phil Davis f0e17fd9c0 libtypos comment and message typos for master 2015-10-05 09:05:09 +05:45
Daniel Molkentin 66e8aaeabc Use doxygen style everywhere 2015-06-29 18:56:09 +02:00
Daniel Molkentin 0735aa1fbd Structure developer documentation
- rename target "doc-dev"
- group into modules
- move to doc/dev
2015-06-29 18:43:21 +02:00
Olivier Goffart 6b27d2e703 Merge remote-tracking branch 'origin/1.8' 2015-05-26 12:28:31 +02:00
Klaas Freitag 8f277e46d6 TransmissionChecksumming: Remove SyncFileItem dependency. 2015-05-21 15:51:48 +02:00
Klaas Freitag 5b5a636cc1 Make TransmissionChecksumValidator child of the job for auto delete.
That way no explicit memory management is needed as the allocation
is freed when the job (parent) is deleted automatically.
2015-05-20 16:54:04 +02:00
Klaas Freitag c7f759fedf Propagator: Use the TransmissionChecksumValidator class. 2015-05-19 17:09:40 +02:00
Klaas Freitag 3701fbcbfe PropagateUpload: Add checksum calculation if required by config.
If the config file has an transmissionChecksum entry, a checksum
is added to the PUT requests in a header.
2015-05-19 17:09:39 +02:00
Daniel Molkentin b54f2b3b31 Merge remote-tracking branch 'origin/1.9'
Conflicts:
	VERSION.cmake
	src/libsync/propagatedownload.cpp
	src/libsync/propagateupload.cpp
2015-05-12 10:03:38 +02:00
Jocelyn Turcotte 1053153ec4 Fix the Qt4 build
Following a6500d8068
Qt4's moc doesn't expand macros and wouldn't evaluate QT_VERSION_CHECK.
2015-05-07 14:49:01 +02:00
Jocelyn Turcotte a6500d8068 Work around the Qt PUT corruption bug with Qt < 5.4.2 #2425
Since QNonContiguousByteDeviceThreadForwardImpl::reset will
call UploadDevice::reset with a BlockingQueuedConnection, this
allows us to reset the HTTP channel along with its buffers
before they get the chance to be reused with a subsequent request.
2015-05-06 16:56:50 +02:00
Christian Kamm 509b83e73e Time estimation: Avoid a progress reset before finish. #2328
The current algorithm doesn't care much, but resetting progress
to 0 just before completing a job is confusing anyway.
2015-04-22 10:45:41 +02:00
Jocelyn Turcotte 12d7484b85 Pass SyncFileItems by pointer instead of by value
Use a QSharedPointer to keep the same ownership and
continue passing the SyncFileItems as a const& when
ownership isn't taken. This allows sharing the same
allocations between the jobs and the result vectors.

This saves about 20MB of memory (off 120MB) once all
jobs are created.
2015-04-18 14:26:40 +02:00
Jocelyn Turcotte 3c9acdf724 Fix a crash when accessing a dangling UploadDevice pointer #2984
This is largely a guess, but this is the only place where we use
a QIODevice to push data through QNAM and that the QIODevice isn't
a direct child of the QNetworkReply.

Fix the issue by making sure that we don't go back to the event loop
and possibly handle network events between the destruction of the
upload QIODevice and the QNetworkReply, which might lead to QNAM
dereferencing a dangling QIODevice pointer.
2015-03-23 16:14:54 +01:00
Olivier Goffart 97c221d860 PropagateUpload: add a few comments 2015-03-04 12:00:55 +01:00
Christian Kamm 0af5574951 Propagate upload: code cleanup 2015-01-14 15:20:02 +01:00
Markus Goetz a544133d73 Propagator: Guard access to BandwidthManager
Shouls fix #2623
2015-01-14 14:28:47 +01:00
Olivier Goffart 3ec19ee355 Propatage upload: Preload the chunk in memory and close the file
This should solve #2675 and #1981

By preloading the chunks in memory before sending them, we don't keep the
file open and therefore we let other program open the file for writing.

If the file is modified between two chunks, we detect that and abort anyway
2015-01-14 13:46:51 +01:00
Olivier Goffart 0215b250af UploadDevice: mark overrided method as Q_DECL_OVERRIDE 2015-01-14 13:46:51 +01:00
Olivier Goffart 7f1593c5d7 PropagateUpload: The PUTJob owns the chunk device, so make it a QScopedPointer instead of a QSharedPointer 2015-01-14 13:46:51 +01:00
Christian Kamm 38ebfec1fb Use global Account/AccountState less.
* Use a shared pointer to Account everywhere to ensure
  the instance stays alive long enough for a sync to terminate
* Folder is now tied to an AccountState
* SyncEngine and OwncloudPropagator tie to an Account and use that
  for all jobs they run

Issue: Since the setup wizard currently always replaces the
account, it will always wipe all folder definitions, even when
the actual changes to the account were minor.
2014-12-18 15:39:51 +01:00
Daniel Molkentin 3016844dd7 Merge branch 'master' into rename_client
Conflicts:
	README.md
	src/gui/folderman.cpp
	src/gui/settingsdialog.cpp
	src/libsync/accessmanager.cpp
	src/libsync/propagateupload.h
2014-12-02 13:37:22 +01:00
Olivier Goffart ad92e7b888 Move UploadDevice to the right header 2014-11-14 16:40:13 +01:00
Olivier Goffart c73522de2e Remove ChunkBlock
It is not in used
2014-11-14 16:39:39 +01:00
Daniel Molkentin 281c0e1553 Merge branch 'master' into rename_client
Conflicts:
	CMakeLists.txt
	src/gui/main.cpp
	src/libsync/accessmanager.cpp
	src/libsync/accessmanager.h
	src/libsync/owncloudpropagator_p.h
2014-11-12 00:07:59 +01:00
Olivier Goffart 506ba022f1 Split propagator_qnam.cpp into propagateupload.cpp and propagatedownload.cpp 2014-11-11 12:17:10 +01:00