1
0
mirror of https://github.com/chylex/Brotli-Builder.git synced 2024-12-22 16:42:46 +01:00
Commit Graph

229 Commits

Author SHA1 Message Date
271b43630e Implement MultiTrie.Node caching for Brotli dictionary index 2019-10-05 08:14:47 +02:00
8ecdd90caa Fix recently introduced crash w/ implicit distance code zero
Caused by d8a6713392
2019-10-05 04:19:24 +02:00
3294ac800e Improve how CR/LF characters are displayed in markers 2019-10-04 16:47:55 +02:00
5b44cefe68 Update markers to include backreference & dictionary output 2019-10-04 16:18:05 +02:00
eb3833e4da Fix off-by-x bug when generating dictionary references after hitting window size 2019-10-04 15:30:15 +02:00
d8a6713392 Fix broken generation of explicit distance codes 0 2019-10-04 15:15:27 +02:00
e18cede896 Add new Brotli dictionary index system 2019-10-03 21:56:17 +02:00
9da921fe52 Remove inefficient dictionary index 2019-10-03 17:17:21 +02:00
806c97748a Fix Brotli insert&copy command serialization not picking distance codes efficiently 2019-07-09 15:55:27 +02:00
0b7aaec69d Optimize complex distance codes (with no postfix) w/ unit tests 2019-07-09 15:00:21 +02:00
50035519f8 Make most marker values lazy 2019-07-08 23:46:26 +02:00
c3b66b50e9 Tweaks for minor performance improvements 2019-07-08 23:36:10 +02:00
05690a1bea Improve marker memory usage by interning strings 2019-07-08 18:13:11 +02:00
895609c3f1 Use dummy writer to improve performance in transformations 2019-06-18 15:13:09 +02:00
da910cc147 Refactor BitWriter, BitReader, and MarkedBitReader into interfaces 2019-06-18 15:06:39 +02:00
8c5ae6a960 Separate IBitSerializer into serializer & deserializer delegates 2019-06-18 05:00:33 +02:00
52d435dece Fix invalid complex Huffman tree length codes generated for incomplete trees 2019-05-24 14:11:52 +02:00
6baa782fb9 Make DistanceParameters a readonly struct 2019-05-23 16:24:40 +02:00
82f212a4b8 Add marker for Brotli block-switch type value 2019-05-23 15:30:12 +02:00
1370231216 Add utility Set methods to ContextMap builder 2019-05-23 15:29:43 +02:00
74969f8c92 Implement Brotli block-switch command builder 2019-05-23 15:28:30 +02:00
f289bb2660 Update CompressedMetaBlockBuilder to use FrequencyLists 2019-05-23 14:18:55 +02:00
b90240c180 Update HuffmanTree.FromSymbols to use FrequencyList 2019-05-23 14:15:21 +02:00
62b5236455 Remove category parameter from CategoryMap.Select 2019-05-23 13:11:52 +02:00
9ab0520997 Determine BlockSwitchCommand type codes during serialization instead of storing them 2019-05-23 11:10:59 +02:00
e39ddf5e27 Make ContextMap builder clone the final array 2019-05-20 09:48:05 +02:00
7e7b58e533 Fix broken implicit distance code 0 and dictionary distances in CompressedMetaBlockBuilder 2019-05-19 23:18:04 +02:00
d4fa203f4f Add MetaBlockSizeTracker utility for meta-block size minimization 2019-05-18 02:41:44 +02:00
e349cf112b Move byte array slicing/cloning into a utility CollectionHelper class 2019-05-18 00:38:48 +02:00
35ca0e1fb8 Move existing Brotli encoder & transformer implementations to BrotliImpl 2019-05-17 23:06:41 +02:00
143b79aecf Add convenience method for dictionary references to CompressedMetaBlockBuilder 2019-05-17 18:54:55 +02:00
dee0e6c240 Add DictionaryIndexEntry & refactor index serialization 2019-05-17 18:54:55 +02:00
417b2b0eed Add IMultiTrie, move tries to different namespace, revert trie value sorting 2019-05-17 18:54:55 +02:00
34d801a70b Slightly improve strategy for picking Brotli distance codes 2019-05-17 06:35:26 +02:00
304fa1dd7c Implement Brotli dictionary trie serialization & include word length in index 2019-05-17 03:41:10 +02:00
78052590df Split Brotli distance codes into multiple files 2019-05-16 21:01:18 +02:00
768f2e9ac0 Work on Brotli component validation, constants, and minor refactoring 2019-05-16 19:09:58 +02:00
632f94eef9 Refactor Brotli element construction to guard mutability 2019-05-16 18:25:30 +02:00
d85badda47 Add a few TODOs 2019-04-22 22:59:15 +02:00
6b169a66bd Add a simple transformer to rebuild Brotli files made by official compressor 2019-04-22 17:07:13 +02:00
f736e837e3 Update EncodeLiterals for the updated interfaces 2019-04-22 17:02:19 +02:00
22640d473b Add IBrotliTransformer and utilities for transforming Brotli structures 2019-04-22 16:52:10 +02:00
c6c55fcc97 Automatically fixup structures from IBrotliEncoder so encoders don't have to 2019-04-22 16:47:32 +02:00
b38f6c3485 Implement compressed meta-block reconstruction and chained building 2019-04-22 16:45:30 +02:00
d6c3b5b743 Allow cloning global Brotli state 2019-04-22 16:45:30 +02:00
0d9e7f91f2 Add block-switch command support in CompressedMetaBlockBuilder 2019-04-22 16:45:30 +02:00
d78202b3a5 Fix broken encoding of direct distances 2019-03-28 20:10:59 +01:00
230b4c95b1 Add BrotliFileParameters as a parameter to IBrotliEncoder 2019-03-28 04:26:23 +01:00
18d8f1bff0 Replace references to window size + dictionary w/ BrotliFileParameters 2019-03-28 04:20:08 +01:00
caf2ca9c19 Implement window output strategy for when full output isn't necessary 2019-03-28 04:06:11 +01:00
e7b8d4ce20 Move BrotliGlobalState into a different namespace 2019-03-28 04:01:25 +01:00
9b59e6a988 Move output handling from BrotliGlobalState into a new interface 2019-03-28 03:58:31 +01:00
1e552c2ef1 Add EncodeLiterals w/ UI menu item 2019-03-14 17:12:09 +01:00
329b066017 Add WIP class to help construct compressed meta-blocks 2019-03-14 16:46:24 +01:00
63fceb9e6e Move distance context ID calculation & tweak parts of InsertCopyCommand 2019-03-14 16:38:10 +01:00
e74fbb323d Move IBrotliEncoder implementations into a different namespace 2019-03-14 13:40:20 +01:00
17287c1e9b Improve performance of MultiTrie and dictionary transformations 2019-03-14 13:01:10 +01:00
10366592fc Work on Brotli context map construction and utilities 2019-03-14 08:37:30 +01:00
6c4d5dd2c2 Update BrotliDictionary.GenerateLookup to use default Range parameters 2019-03-14 04:54:36 +01:00
b88fe3f6bd Implement a ghetto way to construct Brotli distance codes 2019-03-14 03:49:31 +01:00
c9b28433d3 Move writing copies and dictionary words to BrotliGlobalState 2019-03-13 12:05:17 +01:00
9fdd2a2389 Add WindowSize as a parameter to IBrotliEncoder.GenerateMetaBlocks 2019-03-13 10:44:11 +01:00
4180882a3f Fix BitDepthFormat.GetPackedValue 2019-03-13 10:00:46 +01:00
1226b78736 Fix DistanceCode.Complex.ToString 2019-03-13 08:22:39 +01:00
6db6dc2bc0 Explain complex distance codes & guard constructors 2019-03-13 08:22:02 +01:00
a9131a3efb Add utilities to help constructing Brotli components & general cleanup 2019-03-13 01:17:09 +01:00
1a18808976 Fix typo in InsertCopyLengthCode constructor 2019-03-12 23:51:38 +01:00
4ce74b4c68 Simplify HuffmanTree construction 2019-03-12 22:20:06 +01:00
943c02d89c Add methods to generate Brotli dictionary lookup tries 2019-03-12 09:39:27 +01:00
835cfdcdaf Refactor distance code zero handling to fix distance buffer push logic 2019-03-12 00:17:40 +01:00
4c0083579a Fix distance encoding using wrong trees w/ postfix bits 2019-03-11 23:26:38 +01:00
b760d566bd Minor refactoring 2019-03-11 10:29:11 +01:00
7d34bc8e98 Optimize memory usage of insert&copy commands 2019-03-11 09:27:53 +01:00
64d19061b2 Add a dummy MarkedBitReader to allow disabling markers 2019-03-11 09:16:08 +01:00
dd4d973e57 Make BitReader.Index an integer (BitStream was already limited to 256 MB) 2019-03-11 09:01:18 +01:00
360ed52eba Fix literal context modes breaking across meta-block boundaries 2019-03-11 06:19:53 +01:00
d8746572d4 Add a way to construct insert&copy length codes from length values w/ unit tests 2019-03-09 03:36:41 +01:00
a46cb9457f Remove unused methods from Category and HuffmanNode 2019-03-09 03:16:14 +01:00
607cfeba64 Include category in BlockTypeInfo to refactor serialization contexts 2019-03-09 03:07:37 +01:00
b08283a8db Improve markers for complex Huffman trees 2019-03-09 01:22:58 +01:00
f98f386b4c Add MarkedBitReader.ReadValue w/ huffman tree and value mapper parameters 2019-03-09 01:22:44 +01:00
285612a1bd Cleanup unused methods and imports 2019-03-07 02:55:48 +01:00
cce96a2567 Fix massive performance issues with MarkerNode 2019-03-06 12:51:50 +01:00
95acb299e1 Implement repeat/skip codes in complex Huffman trees 2019-03-04 17:34:41 +01:00
5a6f4b2ede Implement encoding of RLE and IMTF in Brotli context maps 2019-03-04 11:57:30 +01:00
3d511172a7 Implement markers for all Brotli deserializers 2019-03-03 19:53:42 +01:00
46eb57428c Update ToString (DistanceCode, ComplexLengthCode, HuffmanGenerator.Entry) 2019-03-03 19:53:42 +01:00
6e20da13de Enable deserialization with markers in BrotliFileStructure 2019-03-03 18:47:00 +01:00
7044313e04 Add MarkedBitReader and MarkedBitSerializer w/ a few marker types 2019-03-03 18:47:00 +01:00
07d96290b7 Add Literal to represent literal bytes 2019-03-03 17:27:27 +01:00
03c5f040f6 Redo all Equals/GetHashCode/ToString & add a few missing ones 2019-03-02 11:02:36 +01:00
86e25692b8 Implement Brotli compressed meta-block contents and insert&copy command serialization 2019-02-08 10:52:07 +01:00
cf11a7c0f1 Implement Brotli block-switch command serialization & block tracking 2019-02-08 10:52:07 +01:00
5ffb4bf0be Implement Brotli compressed meta-block header serialization 2019-02-08 10:52:07 +01:00
a7b6d7047d Implement Brotli block type and block length code serialization 2019-02-08 09:24:51 +01:00
66b8795135 Reassign ISLAST bit on meta-blocks after modifying their structure 2019-02-07 02:01:10 +01:00
f3295167b8 Implement basic serialization for Brotli context maps 2018-12-29 02:53:45 +01:00
f81568d95a Make AlphabetSize.BitsPerSymbol return a byte & add default max tree depth constant 2018-12-29 02:48:38 +01:00
5d1823acb0 Implement Brotli distance code and distance value serialization 2018-12-28 22:49:01 +01:00
f23359ccd5 Implement Brotli insert&copy length code alphabet and length serialization 2018-12-15 02:20:46 +01:00
791f31e2d9 Implement basic serialization for Brotli Huffman trees w/ unit tests 2018-12-15 02:01:23 +01:00
a8dcf45861 Add utilities for Brotli insert&copy command code categories 2018-12-14 18:36:58 +01:00
24efbdfe74 Fix extra bit when writing padded empty meta-block header 2018-12-14 06:04:16 +01:00
4ce1554e4b Fix missing bit when writing compressed meta-block header 2018-12-13 14:39:09 +01:00
a3e4983bc5 Implement Move-to-front & inverse transformation w/ unit tests 2018-12-12 23:05:23 +01:00
9dcacebfb7 Finish implementing global (de)compression state 2018-12-06 07:07:50 +01:00
6b98e1c2c3 Add default Brotli dictionary file as an embedded resource 2018-12-04 17:24:48 +01:00
8610d6d1cf Add IDictionarySource implementation that can take any Stream 2018-12-04 17:21:51 +01:00
54fe939444 Add default Brotli dictionary format and transformations 2018-12-02 16:17:45 +01:00
61bea5022f Implement bit depth format for unpacking words in a Brotli dictionary 2018-12-02 14:14:21 +01:00
dc63c1355d Implement file and memory representations of a Brotli dictionary 2018-12-02 14:13:22 +01:00
133b9920a6 Add classes and interfaces to describe a Brotli dictionary 2018-12-02 14:12:12 +01:00
359245c652 Implement Brotli dictionary transformation rules and utilities 2018-12-02 13:33:42 +01:00
286f476cfb Add AlphabetSize utility w/ unit tests 2018-11-23 07:45:01 +01:00
3fe5a82239 Implement Brotli literal context mode serialization 2018-11-21 17:01:08 +01:00
87d65a8b6f Include window size in the global (de)compression state 2018-11-21 16:36:13 +01:00
1696b86ba5 Add a utility for BitSerializer to specify no serialization context without nulls 2018-11-21 16:25:28 +01:00
422def48fa Implement Brotli distance parameter serialization 2018-11-18 18:55:24 +01:00
78d93ba9fd Implement Brotli variable length 1-11 bit code serialization 2018-11-18 01:32:11 +01:00
54c68402bb Implement a Brotli encoder that only generates uncompressed meta-blocks 2018-11-17 16:36:04 +01:00
758d437b23 Add interface to use for Brotli encoders 2018-11-17 16:23:14 +01:00
95fd6d3f9a Implement empty meta-block w/ skipped bytes serialization 2018-11-17 13:13:23 +01:00
675008182f Implement uncompressed meta-block serialization 2018-11-17 13:13:23 +01:00
bf78ed23d3 Add a way to handle Brotli state and generate output during serialization 2018-11-16 19:37:57 +01:00
74270f7276 Add BrotliFileStructure class to describe a Brotli-compressed file 2018-11-16 16:38:44 +01:00
a3f666b479 Begin implementing meta-block serialization 2018-11-16 16:31:35 +01:00
be29857899 Add base classes for different types of meta-blocks 2018-11-16 15:22:48 +01:00
abff015bcf Implement Brotli meta-block data length serialization 2018-11-16 14:47:14 +01:00
dd4d34ef0c Implement Brotli window size serialization 2018-11-16 14:11:29 +01:00