aboutsummaryrefslogtreecommitdiffstats
path: root/tools/dicttool/src (follow)
Commit message (Collapse)AuthorAgeFilesLines
* dicttool header to read stream exhaustivelyAdrian Velicu2014-11-111-3/+15
| | | | Change-Id: I50a286c115f5bd6e93763bd2f79031676d6fffd8
* Dicttool to handle unpackaging non-latest version dictsAdrian Velicu2014-11-111-4/+6
| | | | Change-Id: I738735186213b3a40eff997ae2fd83069c6445f1
* Merge "Making 'dicttool header' output format version"Adrian Velicu2014-11-111-0/+1
|\
| * Making 'dicttool header' output format versionAdrian Velicu2014-11-081-0/+1
| | | | | | | | Change-Id: I4198f6b463711feb4ab78020934cca4d23870fbb
* | Move util classes under commonJean Chalard2014-11-071-1/+1
|/ | | | | | | Also why did we have two copies of LocaleUtils >.> Bug: 18108776 Change-Id: I03b4403dfd51934e66b567f2f8b87da419cfb3ab
* Add a header command to dicttool.Jean Chalard2014-11-063-10/+92
| | | | | | | | | | | | This will allow to greatly improve the performance of the metadata-generating files, as they won't have to wait for the info command to read the entire dictionary when the header is all we need. Also add tests, and while we're at it, use the seed as intended to enable reproducible tests. Change-Id: I0ba79ef62f0292b23e63aed57ff565bb102281a2
* Add a *FAST* dictionary header reader.Jean Chalard2014-11-061-0/+53
| | | | | | | | | | | It's still unused as of this change but the next change will use it As a reference point, generating the metadata for Bayo takes 3'02" on my machine with the info command; it's down to 16" if made to use this instead. The gains increases with the number of dictionaries obviously. Change-Id: I0eeea2d8f81bb74b0d1570af658e91b56f7c2b79
* Genericize getting a raw dictionaryJean Chalard2014-11-062-45/+68
| | | | | | | This will allow for not copying the whole dictionary when only the header is needed. Change-Id: Ie4a649b507ccd4a430201824ed87b8b8bbf55e9f
* Large simplification in obtaining a raw dictionaryJean Chalard2014-11-051-81/+59
| | | | | | | | | | | That is where the last refactorings were leading. This code is simpler, but it's far more flexible. Importantly, it only makes a single copy instead of making a full disk copy for every intermediate step. Next we're going to make the "copy" part modular for processes that don't need to copy the whole file. Change-Id: Ief32ac665d804b9b20c44f443a9c87452ceb367a
* Merge "Initial commit for native dicttoolkit."Keisuke Kuroyanagi2014-10-311-0/+1
|\
| * Initial commit for native dicttoolkit.Keisuke Kuroyanagi2014-10-311-0/+1
| | | | | | | | | | | | Bug: 10059681 Change-Id: Ib730af8ebc944e08aaada869c0626724a499747c
* | Using "blacklist" flag as "possibly offensive"Adrian Velicu2014-10-312-21/+32
|/ | | | | Bug: 11031090 Change-Id: I5cc0d006ab003656498eb82b0875eb9c051d331e
* Fix unit test breakageTadashi G. Takaoka2014-10-241-1/+0
| | | | Change-Id: I538288054a58eb2c81ce3cbe5c9bef900fb653a5
* Merge "Some more simplification of DecoderSpec works"Jean Chalard2014-10-241-6/+13
|\
| * Some more simplification of DecoderSpec worksJean Chalard2014-10-231-6/+13
| | | | | | | | Change-Id: I23fa4e4ed96228406e70aa94d84fd7b8d3f69347
* | Merge "Simplify handling of steps in DecoderChainSpec"Jean Chalard2014-10-241-12/+25
|\|
| * Simplify handling of steps in DecoderChainSpecJean Chalard2014-10-231-12/+25
| | | | | | | | | | | | | | This is a preliminary refactoring change to improve performance in dicttool diagnostic tools. Change-Id: I9a59328af62e336809246be5bebbbf2e154366b3
* | Remove unused import and methodTadashi G. Takaoka2014-10-231-9/+0
|/ | | | | Bug: 18003991 Change-Id: Id6b67bf66b397301e5186826dba2b60df9cb4c65
* Fix Javadoc and null analysis related warningsTadashi G. Takaoka2014-10-232-3/+3
| | | | | | | This CL also adds @SuppressWarning("unused" to java-overridable package. Bug: 18003991 Change-Id: If70527e30654384705d7a814f5efd181d9f539e1
* Remove XML input/output from dicttool.Jean Chalard2014-10-224-503/+13
| | | | | | This hasn't been used for a while. It's deprecated. Let's kill it. Change-Id: Ib1c491fa14b6406f6f77f2b0869f4db1810eb078
* Fix some compiler warningsTadashi G. Takaoka2014-10-212-4/+1
| | | | | | | | | | | | | | | | | | | | | This CL fixes the following compiler warnings. - Indirect access to static member - Access to a non-accessible member of an enclosing type - Parameter assignment - Method can be static - Local variable declaration hides another field or variable - Value of local variable is not used - Unused import - Unused private member - Unnecessary 'else' statement - Unnecessary declaration of throw exception - Redundant type arguments - Missing '@Override' annotation - Unused '@SuppressWarning' annotations Bug: 18003991 Change-Id: Icfebe753e53a2cc621848f769d6a3d7ce501ebc7
* Renaming "blacklist" flag to "possibly offensive"Adrian Velicu2014-10-214-10/+12
| | | | | | | | | No behaviour changes. Unified the overloaded FusionDictionary::add method to always take an isPossiblyOffensive argument. Bug: 11031090 Change-Id: I5741a023ca1ce842d2cf10d4f6c926b0efabaa78
* Merge "Include a code point table in the binary dictionary."Akifumi Yoshimoto2014-10-021-1/+1
|\
| * Include a code point table in the binary dictionary.Akifumi Yoshimoto2014-10-021-1/+1
| | | | | | | | | | Bug:17097992 Change-Id: I677a5eb3a704e4386f6573360e44ca335d81d2df
* | Introduce NgramProperty in Java side.Keisuke Kuroyanagi2014-10-013-6/+6
|/ | | | | Bug: 14425059 Change-Id: I8b3458ad22730b3dccbe0caea2c5930f5276dc82
* Read dicttool option for switching code point tableAkifumi Yoshimoto2014-09-261-17/+36
| | | | | Bug:17097992 Change-Id: I0b3f12c4450f784b9a33470d1dc4c306062de91e
* Refactor dicttool with try-with-resourceTadashi G. Takaoka2014-06-227-168/+182
| | | | | | This CL must be checked in together with Idd7c744d0f. Change-Id: Ia0ff09a054c1852b39cdce22a4377108afb254e2
* Use Java 7 diamond operatorTadashi G. Takaoka2014-05-247-20/+17
| | | | Change-Id: If16ef50ae73147594615d0f49d6a22621eaf1aef
* Have dicttool test tidy up after itself.Jean Chalard2014-05-231-3/+29
| | | | | Bug: 13776363 Change-Id: Icb1d3fc0efe71e0339b434928e8aed507f2fb590
* Move code only used for dicttool and tests under tests.Keisuke Kuroyanagi2014-03-273-47/+6
| | | | | Bug: 13035567 Change-Id: I13c6df013ef2b67c9bf67455d9c32d283bf9ea2e
* Fix: dicttool build.Keisuke Kuroyanagi2014-03-071-0/+1
| | | | Change-Id: I5c3bcbe9f3054bdd1a760398fe11344e0e05ac6a
* Move DictionaryOptions from FusionDictionary to FormatSpec.Keisuke Kuroyanagi2014-03-062-2/+1
| | | | | | | Bug: 8187060 Bug:13035567 Change-Id: Id4f45e589521ae98c926a4c0607be10ce1a983f2
* Separate WeightedString from FusionDictionary.Keisuke Kuroyanagi2014-03-064-4/+4
| | | | | | Bug: 8187060 Change-Id: I40c1dafca3eb52244c64fdb4c1db30a56385d678
* Fix: dicttool build.Keisuke Kuroyanagi2014-02-281-0/+22
| | | | Change-Id: I592b14eba895786d0981586a01ef545e003396c8
* Correctly read the header of APK-embedded dictsJean Chalard2014-02-242-2/+2
| | | | | Bug: 13164518 Change-Id: I8768ad887af8b89ad9f29637f606c3c68629c7ca
* Remove unused argument from readDictionaryBinary.Keisuke Kuroyanagi2014-02-142-2/+2
| | | | | Bug: 12810574 Change-Id: Ice415ebd8d11162facca3fe8927ef8a616b11424
* Remove unused code.Keisuke Kuroyanagi2014-02-101-1/+0
| | | | | Bug: 12810574 Change-Id: If0ef02a984469a3b6e0c00b1c3c8d98d0d2b5466
* Make PtNode have ProbabilityInfo instead of raw value.Keisuke Kuroyanagi2014-02-104-17/+35
| | | | | | Bug: 11281877 Bug: 12810574 Change-Id: Id1cda0afc74c4e30633c735729143491b2274a7b
* Use CombinedFormatUtils to convert dict elements to strings.Keisuke Kuroyanagi2014-02-061-48/+34
| | | | | | Bug: 11281877 Bug: 12810574 Change-Id: Ib631f75eab73abc9877a7698171c45e8f2fc7600
* Consolidate WordProperty and Word.Keisuke Kuroyanagi2014-02-064-65/+71
| | | | | | Bug: 11281877 Bug: 12810574 Change-Id: I9dc99188f80f25a8780c1860dab46e4aa80a23e5
* Make WeightedString have ProbabilityInfo.Keisuke Kuroyanagi2014-02-054-16/+20
| | | | | | Bug: 11281877 Bug: 12810574 Change-Id: I265e3d8654c75766cd0e0d09d67ef62b4566298a
* Remove ver4 dict updater.Keisuke Kuroyanagi2013-12-171-3/+2
| | | | Change-Id: I468994c98d091be621b9fb3fbe6405c67fc6a465
* Fix failing testsJean Chalard2013-12-161-3/+0
| | | | | | | - Version 3 is not supported - Now passing the right string to open v4 dicts. Fix the tests for this. Change-Id: I7829330c3568a715b96396ba4e4e69c6e17775ab
* Have dicttool use the native library to generate v4 dicts.Jean Chalard2013-12-131-2/+2
| | | | | | Yay ! Change-Id: Iea8ced9e81031b9ab7eff05ad9ef7215be248de9
* Remove flags from Java side.Jean Chalard2013-12-133-37/+3
| | | | | | | | | | | | This simplifies the code quite a bit. - GERMAN_UMLAUTS are now handled through a key-value attribute. The dictionary generator does not need to know about it any more. - FRENCH_LIGATURES are deprecated as we handle them with shortcuts now. - CONTAINS_BIGRAMS is deprecated. Bigram processing is always applied regardless of this flag. Bug: 11281748 Change-Id: If567e52e245a9342adc7f3104a0f7d8d782df8c1
* Reset to 9bd6dac4708ad94fd0257c53e977df62b152e20cKen Wakasa2013-12-131-3/+3
| | | | | | The bulk merge from -bayo to klp-dev should not have been merged to master. Change-Id: I527a03a76f5247e4939a672f27c314dc11cbb854
* Show more messages when reading a compressed combined format file.Yuichiro Hanada2013-10-081-1/+5
| | | | Change-Id: I51a1b9454fcfe656e0fcf762dcfd9ecbadde86c3
* Make dicttool read the compressed combined format.Yuichiro Hanada2013-09-302-15/+17
| | | | Change-Id: Ib39fa110402895a655f4e705caae53397ace9259
* Fix getDictionary.Yuichiro Hanada2013-09-261-1/+1
| | | | Change-Id: I6bc3ec8dd4397a9aaf9dca2f16ce8a1929a47e9e
* Add a new option for version 4 to dicttool.Yuichiro Hanada2013-09-191-7/+12
| | | | Change-Id: I18fd48c1f6921758d30330fbc77f4a917c33f1c8
generated by cgit v1.2.3-83-g751a (git 2.47.1) at 2025-05-04 22:55:13 +0000