forked from pyparsing/pyparsing
-
Notifications
You must be signed in to change notification settings - Fork 0
/
CHANGES
2597 lines (1965 loc) · 106 KB
/
CHANGES
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
==========
Change Log
==========
Version 2.3.1 - January, 2019
-----------------------------
- POSSIBLE API CHANGE: this release fixes a bug when results names were
attached to a MatchFirst or Or object containing an And object.
Previously, a results name on an And object within an enclosing MatchFirst
or Or could return just the first token in the And. Now, all the tokens
matched by the And are correctly returned. This may result in subtle
changes in the tokens returned if you have this condition in your pyparsing
scripts.
- New staticmethod ParseException.explain() to help diagnose parse exceptions
by showing the failing input line and the trace of ParserElements in
the parser leading up to the exception. explain() returns a multiline
string listing each element by name. (This is still an experimental
method, and the method signature and format of the returned string may
evolve over the next few releases.)
Example:
# define a parser to parse an integer followed by an
# alphabetic word
expr = pp.Word(pp.nums).setName("int")
+ pp.Word(pp.alphas).setName("word")
try:
# parse a string with a numeric second value instead of alpha
expr.parseString("123 355")
except pp.ParseException as pe:
print_(pp.ParseException.explain(pe))
Prints:
123 355
^
ParseException: Expected word (at char 4), (line:1, col:5)
__main__.ExplainExceptionTest
pyparsing.And - {int word}
pyparsing.Word - word
explain() will accept any exception type and will list the function
names and parse expressions in the stack trace. This is especially
useful when an exception is raised in a parse action.
Note: explain() is only supported under Python 3.
- Fix bug in dictOf which could match an empty sequence, making it
infinitely loop if wrapped in a OneOrMore.
- Added unicode sets to pyparsing_unicode for Latin-A and Latin-B ranges.
- Added ability to define custom unicode sets as combinations of other sets
using multiple inheritance.
class Turkish_set(pp.pyparsing_unicode.Latin1, pp.pyparsing_unicode.LatinA):
pass
turkish_word = pp.Word(Turkish_set.alphas)
- Updated state machine import examples, with state machine demos for:
. traffic light
. library book checkin/checkout
. document review/approval
In the traffic light example, you can use the custom 'statemachine' keyword
to define the states for a traffic light, and have the state classes
auto-generated for you:
statemachine TrafficLightState:
Red -> Green
Green -> Yellow
Yellow -> Red
Similar for state machines with named transitions, like the library book
state example:
statemachine LibraryBookState:
New -(shelve)-> Available
Available -(reserve)-> OnHold
OnHold -(release)-> Available
Available -(checkout)-> CheckedOut
CheckedOut -(checkin)-> Available
Once the classes are defined, then additional Python code can reference those
classes to add class attributes, instance methods, etc.
See the examples in examples/statemachine
- Added an example parser for the decaf language. This language is used in
CS compiler classes in many colleges and universities.
- Fixup of docstrings to Sphinx format, inclusion of test files in the source
package, and convert markdown to rst throughout the distribution, great job
by Matěj Cepl!
- Expanded the whitespace characters recognized by the White class to include
all unicode defined spaces. Suggested in Issue #51 by rtkjbillo.
- Added optional postParse argument to ParserElement.runTests() to add a
custom callback to be called for test strings that parse successfully. Useful
for running tests that do additional validation or processing on the parsed
results. See updated chemicalFormulas.py example.
- Removed distutils fallback in setup.py. If installing the package fails,
please update to the latest version of setuptools. Plus overall project code
cleanup (CRLFs, whitespace, imports, etc.), thanks Jon Dufresne!
- Fix bug in CaselessKeyword, to make its behavior consistent with
Keyword(caseless=True). Fixes Issue #65 reported by telesphore.
Version 2.3.0 - October, 2018
-----------------------------
- NEW SUPPORT FOR UNICODE CHARACTER RANGES
This release introduces the pyparsing_unicode namespace class, defining
a series of language character sets to simplify the definition of alphas,
nums, alphanums, and printables in the following language sets:
. Arabic
. Chinese
. Cyrillic
. Devanagari
. Greek
. Hebrew
. Japanese (including Kanji, Katakana, and Hirigana subsets)
. Korean
. Latin1 (includes 7 and 8-bit Latin characters)
. Thai
. CJK (combination of Chinese, Japanese, and Korean sets)
For example, your code can define words using:
korean_word = Word(pyparsing_unicode.Korean.alphas)
See their use in the updated examples greetingInGreek.py and
greetingInKorean.py.
This namespace class also offers access to these sets using their
unicode identifiers.
- POSSIBLE API CHANGE: Fixed bug where a parse action that explicitly
returned the input ParseResults could add another nesting level in
the results if the current expression had a results name.
vals = pp.OneOrMore(pp.pyparsing_common.integer)("int_values")
def add_total(tokens):
tokens['total'] = sum(tokens)
return tokens # this line can be removed
vals.addParseAction(add_total)
print(vals.parseString("244 23 13 2343").dump())
Before the fix, this code would print (note the extra nesting level):
[244, 23, 13, 2343]
- int_values: [244, 23, 13, 2343]
- int_values: [244, 23, 13, 2343]
- total: 2623
- total: 2623
With the fix, this code now prints:
[244, 23, 13, 2343]
- int_values: [244, 23, 13, 2343]
- total: 2623
This fix will change the structure of ParseResults returned if a
program defines a parse action that returns the tokens that were
sent in. This is not necessary, and statements like "return tokens"
in the example above can be safely deleted prior to upgrading to
this release, in order to avoid the bug and get the new behavior.
Reported by seron in Issue #22, nice catch!
- POSSIBLE API CHANGE: Fixed a related bug where a results name
erroneously created a second level of hierarchy in the returned
ParseResults. The intent for accumulating results names into ParseResults
is that, in the absence of Group'ing, all names get merged into a
common namespace. This allows us to write:
key_value_expr = (Word(alphas)("key") + '=' + Word(nums)("value"))
result = key_value_expr.parseString("a = 100")
and have result structured as {"key": "a", "value": "100"}
instead of [{"key": "a"}, {"value": "100"}].
However, if a named expression is used in a higher-level non-Group
expression that *also* has a name, a false sub-level would be created
in the namespace:
num = pp.Word(pp.nums)
num_pair = ("[" + (num("A") + num("B"))("values") + "]")
U = num_pair.parseString("[ 10 20 ]")
print(U.dump())
Since there is no grouping, "A", "B", and "values" should all appear
at the same level in the results, as:
['[', '10', '20', ']']
- A: '10'
- B: '20'
- values: ['10', '20']
Instead, an extra level of "A" and "B" show up under "values":
['[', '10', '20', ']']
- A: '10'
- B: '20'
- values: ['10', '20']
- A: '10'
- B: '20'
This bug has been fixed. Now, if this hierarchy is desired, then a
Group should be added:
num_pair = ("[" + pp.Group(num("A") + num("B"))("values") + "]")
Giving:
['[', ['10', '20'], ']']
- values: ['10', '20']
- A: '10'
- B: '20'
But in no case should "A" and "B" appear in multiple levels. This bug-fix
fixes that.
If you have current code which relies on this behavior, then add or remove
Groups as necessary to get your intended results structure.
Reported by Athanasios Anastasiou.
- IndexError's raised in parse actions will get explicitly reraised
as ParseExceptions that wrap the original IndexError. Since
IndexError sometimes occurs as part of pyparsing's normal parsing
logic, IndexErrors that are raised during a parse action may have
gotten silently reinterpreted as parsing errors. To retain the
information from the IndexError, these exceptions will now be
raised as ParseExceptions that reference the original IndexError.
This wrapping will only be visible when run under Python3, since it
emulates "raise ... from ..." syntax.
Addresses Issue #4, reported by guswns0528.
- Added Char class to simplify defining expressions of a single
character. (Char("abc") is equivalent to Word("abc", exact=1))
- Added class PrecededBy to perform lookbehind tests. PrecededBy is
used in the same way as FollowedBy, passing in an expression that
must occur just prior to the current parse location.
For fixed-length expressions like a Literal, Keyword, Char, or a
Word with an `exact` or `maxLen` length given, `PrecededBy(expr)`
is sufficient. For varying length expressions like a Word with no
given maximum length, `PrecededBy` must be constructed with an
integer `retreat` argument, as in
`PrecededBy(Word(alphas, nums), retreat=10)`, to specify the maximum
number of characters pyparsing must look backward to make a match.
pyparsing will check all the values from 1 up to retreat characters
back from the current parse location.
When stepping backwards through the input string, PrecededBy does
*not* skip over whitespace.
PrecededBy can be created with a results name so that, even though
it always returns an empty parse result, the result *can* include
named results.
Idea first suggested in Issue #30 by Freakwill.
- Updated FollowedBy to accept expressions that contain named results,
so that results names defined in the lookahead expression will be
returned, even though FollowedBy always returns an empty list.
Inspired by the same feature implemented in PrecededBy.
Version 2.2.2 - September, 2018
-------------------------------
- Fixed bug in SkipTo, if a SkipTo expression that was skipping to
an expression that returned a list (such as an And), and the
SkipTo was saved as a named result, the named result could be
saved as a ParseResults - should always be saved as a string.
Issue #28, reported by seron.
- Added simple_unit_tests.py, as a collection of easy-to-follow unit
tests for various classes and features of the pyparsing library.
Primary intent is more to be instructional than actually rigorous
testing. Complex tests can still be added in the unitTests.py file.
- New features added to the Regex class:
- optional asGroupList parameter, returns all the capture groups as
a list
- optional asMatch parameter, returns the raw re.match result
- new sub(repl) method, which adds a parse action calling
re.sub(pattern, repl, parsed_result). Simplifies creating
Regex expressions to be used with transformString. Like re.sub,
repl may be an ordinary string (similar to using pyparsing's
replaceWith), or may contain references to capture groups by group
number, or may be a callable that takes an re match group and
returns a string.
For instance:
expr = pp.Regex(r"([Hh]\d):\s*(.*)").sub(r"<\1>\2</\1>")
expr.transformString("h1: This is the title")
will return
<h1>This is the title</h1>
- Fixed omission of LICENSE file in source tarball, also added
CODE_OF_CONDUCT.md per GitHub community standards.
Version 2.2.1 - September, 2018
-------------------------------
- Applied changes necessary to migrate hosting of pyparsing source
over to GitHub. Many thanks for help and contributions from hugovk,
jdufresne, and cngkaygusuz among others through this transition,
sorry it took me so long!
- Fixed import of collections.abc to address DeprecationWarnings
in Python 3.7.
- Updated oc.py example to support function calls in arithmetic
expressions; fixed regex for '==' operator; and added packrat
parsing. Raised on the pyparsing wiki by Boris Marin, thanks!
- Fixed bug in select_parser.py example, group_by_terms was not
reported. Reported on SF bugs by Adam Groszer, thanks Adam!
- Added "Getting Started" section to the module docstring, to
guide new users to the most common starting points in pyparsing's
API.
- Fixed bug in Literal and Keyword classes, which erroneously
raised IndexError instead of ParseException.
Version 2.2.0 - March, 2017
---------------------------
- Bumped minor version number to reflect compatibility issues with
OneOrMore and ZeroOrMore bugfixes in 2.1.10. (2.1.10 fixed a bug
that was introduced in 2.1.4, but the fix could break code
written against 2.1.4 - 2.1.9.)
- Updated setup.py to address recursive import problems now
that pyparsing is part of 'packaging' (used by setuptools).
Patch submitted by Joshua Root, much thanks!
- Fixed KeyError issue reported by Yann Bizeul when using packrat
parsing in the Graphite time series database, thanks Yann!
- Fixed incorrect usages of '\' in literals, as described in
https://docs.python.org/3/whatsnew/3.6.html#deprecated-python-behavior
Patch submitted by Ville Skyttä - thanks!
- Minor internal change when using '-' operator, to be compatible
with ParserElement.streamline() method.
- Expanded infixNotation to accept a list or tuple of parse actions
to attach to an operation.
- New unit test added for dill support for storing pyparsing parsers.
Ordinary Python pickle can be used to pickle pyparsing parsers as
long as they do not use any parse actions. The 'dill' module is an
extension to pickle which *does* support pickling of attached
parse actions.
Version 2.1.10 - October, 2016
-------------------------------
- Fixed bug in reporting named parse results for ZeroOrMore
expressions, thanks Ethan Nash for reporting this!
- Fixed behavior of LineStart to be much more predictable.
LineStart can now be used to detect if the next parse position
is col 1, factoring in potential leading whitespace (which would
cause LineStart to fail). Also fixed a bug in col, which is
used in LineStart, where '\n's were erroneously considered to
be column 1.
- Added support for multiline test strings in runTests.
- Fixed bug in ParseResults.dump when keys were not strings.
Also changed display of string values to show them in quotes,
to help distinguish parsed numeric strings from parsed integers
that have been converted to Python ints.
Version 2.1.9 - September, 2016
-------------------------------
- Added class CloseMatch, a variation on Literal which matches
"close" matches, that is, strings with at most 'n' mismatching
characters.
- Fixed bug in Keyword.setDefaultKeywordChars(), reported by Kobayashi
Shinji - nice catch, thanks!
- Minor API change in pyparsing_common. Renamed some of the common
expressions to PEP8 format (to be consistent with the other
pyparsing_common expressions):
. signedInteger -> signed_integer
. sciReal -> sci_real
Also, in trying to stem the API bloat of pyparsing, I've copied
some of the global expressions and helper parse actions into
pyparsing_common, with the originals to be deprecated and removed
in a future release:
. commaSeparatedList -> pyparsing_common.comma_separated_list
. upcaseTokens -> pyparsing_common.upcaseTokens
. downcaseTokens -> pyparsing_common.downcaseTokens
(I don't expect any other expressions, like the comment expressions,
quotedString, or the Word-helping strings like alphas, nums, etc.
to migrate to pyparsing_common - they are just too pervasive. As for
the PEP8 vs camelCase naming, all the expressions are PEP8, while
the parse actions in pyparsing_common are still camelCase. It's a
small step - when pyparsing 3.0 comes around, everything will change
to PEP8 snake case.)
- Fixed Python3 compatibility bug when using dict keys() and values()
in ParseResults.getName().
- After some prodding, I've reworked the unitTests.py file for
pyparsing over the past few releases. It uses some variations on
unittest to handle my testing style. The test now:
. auto-discovers its test classes (while maintining their order
of definition)
. suppresses voluminous 'print' output for tests that pass
Version 2.1.8 - August, 2016
----------------------------
- Fixed issue in the optimization to _trim_arity, when the full
stacktrace is retrieved to determine if a TypeError is raised in
pyparsing or in the caller's parse action. Code was traversing
the full stacktrace, and potentially encountering UnicodeDecodeError.
- Fixed bug in ParserElement.inlineLiteralsUsing, causing infinite
loop with Suppress.
- Fixed bug in Each, when merging named results from multiple
expressions in a ZeroOrMore or OneOrMore. Also fixed bug when
ZeroOrMore expressions were erroneously treated as required
expressions in an Each expression.
- Added a few more inline doc examples.
- Improved use of runTests in several example scripts.
Version 2.1.7 - August, 2016
----------------------------
- Fixed regression reported by Andrea Censi (surfaced in PyContracts
tests) when using ParseSyntaxExceptions (raised when using operator '-')
with packrat parsing.
- Minor fix to oneOf, to accept all iterables, not just space-delimited
strings and lists. (If you have a list or set of strings, it is
not necessary to concat them using ' '.join to pass them to oneOf,
oneOf will accept the list or set or generator directly.)
Version 2.1.6 - August, 2016
----------------------------
- *Major packrat upgrade*, inspired by patch provided by Tal Einat -
many, many, thanks to Tal for working on this! Tal's tests show
faster parsing performance (2X in some tests), *and* memory reduction
from 3GB down to ~100MB! Requires no changes to existing code using
packratting. (Uses OrderedDict, available in Python 2.7 and later.
For Python 2.6 users, will attempt to import from ordereddict
backport. If not present, will implement pure-Python Fifo dict.)
- Minor API change - to better distinguish between the flexible
numeric types defined in pyparsing_common, I've changed "numeric"
(which parsed numbers of different types and returned int for ints,
float for floats, etc.) and "number" (which parsed numbers of int
or float type, and returned all floats) to "number" and "fnumber"
respectively. I hope the "f" prefix of "fnumber" will be a better
indicator of its internal conversion of parsed values to floats,
while the generic "number" is similar to the flexible number syntax
in other languages. Also fixed a bug in pyparsing_common.numeric
(now renamed to pyparsing_common.number), integers were parsed and
returned as floats instead of being retained as ints.
- Fixed bug in upcaseTokens and downcaseTokens introduced in 2.1.5,
when the parse action was used in conjunction with results names.
Reported by Steven Arcangeli from the dql project, thanks for your
patience, Steven!
- Major change to docs! After seeing some comments on reddit about
general issue with docs of Python modules, and thinking that I'm a
little overdue in doing some doc tuneup on pyparsing, I decided to
following the suggestions of the redditor and add more inline examples
to the pyparsing reference documentation. I hope this addition
will clarify some of the more common questions people have, especially
when first starting with pyparsing/Python.
- Deprecated ParseResults.asXML. I've never been too happy with this
method, and it usually forces some unnatural code in the parsers in
order to get decent tag names. The amount of guesswork that asXML
has to do to try to match names with values should have been a red
flag from day one. If you are using asXML, you will need to implement
your own ParseResults->XML serialization. Or consider migrating to
a more current format such as JSON (which is very easy to do:
results_as_json = json.dumps(parse_result.asDict()) Hopefully, when
I remove this code in a future version, I'll also be able to simplify
some of the craziness in ParseResults, which IIRC was only there to try
to make asXML work.
- Updated traceParseAction parse action decorator to show the repr
of the input and output tokens, instead of the str format, since
str has been simplified to just show the token list content.
(The change to ParseResults.__str__ occurred in pyparsing 2.0.4, but
it seems that didn't make it into the release notes - sorry! Too
many users, especially beginners, were confused by the
"([token_list], {names_dict})" str format for ParseResults, thinking
they were getting a tuple containing a list and a dict. The full form
can be seen if using repr().)
For tracing tokens in and out of parse actions, the more complete
repr form provides important information when debugging parse actions.
Verison 2.1.5 - June, 2016
------------------------------
- Added ParserElement.split() generator method, similar to re.split().
Includes optional arguments maxsplit (to limit the number of splits),
and includeSeparators (to include the separating matched text in the
returned output, default=False).
- Added a new parse action construction helper tokenMap, which will
apply a function and optional arguments to each element in a
ParseResults. So this parse action:
def lowercase_all(tokens):
return [str(t).lower() for t in tokens]
OneOrMore(Word(alphas)).setParseAction(lowercase_all)
can now be written:
OneOrMore(Word(alphas)).setParseAction(tokenMap(str.lower))
Also simplifies writing conversion parse actions like:
integer = Word(nums).setParseAction(lambda t: int(t[0]))
to just:
integer = Word(nums).setParseAction(tokenMap(int))
If additional arguments are necessary, they can be included in the
call to tokenMap, as in:
hex_integer = Word(hexnums).setParseAction(tokenMap(int, 16))
- Added more expressions to pyparsing_common:
. IPv4 and IPv6 addresses (including long, short, and mixed forms
of IPv6)
. MAC address
. ISO8601 date and date time strings (with named fields for year, month, etc.)
. UUID (xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx)
. hex integer (returned as int)
. fraction (integer '/' integer, returned as float)
. mixed integer (integer '-' fraction, or just fraction, returned as float)
. stripHTMLTags (parse action to remove tags from HTML source)
. parse action helpers convertToDate and convertToDatetime to do custom parse
time conversions of parsed ISO8601 strings
- runTests now returns a two-tuple: success if all tests succeed,
and an output list of each test and its output lines.
- Added failureTests argument (default=False) to runTests, so that
tests can be run that are expected failures, and runTests' success
value will return True only if all tests *fail* as expected. Also,
parseAll now defaults to True.
- New example numerics.py, shows samples of parsing integer and real
numbers using locale-dependent formats:
4.294.967.295,000
4 294 967 295,000
4,294,967,295.000
Version 2.1.4 - May, 2016
------------------------------
- Split out the '==' behavior in ParserElement, now implemented
as the ParserElement.matches() method. Using '==' for string test
purposes will be removed in a future release.
- Expanded capabilities of runTests(). Will now accept embedded
comments (default is Python style, leading '#' character, but
customizable). Comments will be emitted along with the tests and
test output. Useful during test development, to create a test string
consisting only of test case description comments separated by
blank lines, and then fill in the test cases. Will also highlight
ParseFatalExceptions with "(FATAL)".
- Added a 'pyparsing_common' class containing common/helpful little
expressions such as integer, float, identifier, etc. I used this
class as a sort of embedded namespace, to contain these helpers
without further adding to pyparsing's namespace bloat.
- Minor enhancement to traceParseAction decorator, to retain the
parse action's name for the trace output.
- Added optional 'fatal' keyword arg to addCondition, to indicate that
a condition failure should halt parsing immediately.
Version 2.1.3 - May, 2016
------------------------------
- _trim_arity fix in 2.1.2 was very version-dependent on Py 3.5.0.
Now works for Python 2.x, 3.3, 3.4, 3.5.0, and 3.5.1 (and hopefully
beyond).
Version 2.1.2 - May, 2016
------------------------------
- Fixed bug in _trim_arity when pyparsing code is included in a
PyInstaller, reported by maluwa.
- Fixed catastrophic regex backtracking in implementation of the
quoted string expressions (dblQuotedString, sglQuotedString, and
quotedString). Reported on the pyparsing wiki by webpentest,
good catch! (Also tuned up some other expressions susceptible to the
same backtracking problem, such as cStyleComment, cppStyleComment,
etc.)
Version 2.1.1 - March, 2016
---------------------------
- Added support for assigning to ParseResults using slices.
- Fixed bug in ParseResults.toDict(), in which dict values were always
converted to dicts, even if they were just unkeyed lists of tokens.
Reported on SO by Gerald Thibault, thanks Gerald!
- Fixed bug in SkipTo when using failOn, reported by robyschek, thanks!
- Fixed bug in Each introduced in 2.1.0, reported by AND patch and
unit test submitted by robyschek, well done!
- Removed use of functools.partial in replaceWith, as this creates
an ambiguous signature for the generated parse action, which fails in
PyPy. Reported by Evan Hubinger, thanks Evan!
- Added default behavior to QuotedString to convert embedded '\t', '\n',
etc. characters to their whitespace counterparts. Found during Q&A
exchange on SO with Maxim.
Version 2.1.0 - February, 2016
------------------------------
- Modified the internal _trim_arity method to distinguish between
TypeError's raised while trying to determine parse action arity and
those raised within the parse action itself. This will clear up those
confusing "<lambda>() takes exactly 1 argument (0 given)" error
messages when there is an actual TypeError in the body of the parse
action. Thanks to all who have raised this issue in the past, and
most recently to Michael Cohen, who sent in a proposed patch, and got
me to finally tackle this problem.
- Added compatibility for pickle protocols 2-4 when pickling ParseResults.
In Python 2.x, protocol 0 was the default, and protocol 2 did not work.
In Python 3.x, protocol 3 is the default, so explicitly naming
protocol 0 or 1 was required to pickle ParseResults. With this release,
all protocols 0-4 are supported. Thanks for reporting this on StackOverflow,
Arne Wolframm, and for providing a nice simple test case!
- Added optional 'stopOn' argument to ZeroOrMore and OneOrMore, to
simplify breaking on stop tokens that would match the repetition
expression.
It is a common problem to fail to look ahead when matching repetitive
tokens if the sentinel at the end also matches the repetition
expression, as when parsing "BEGIN aaa bbb ccc END" with:
"BEGIN" + OneOrMore(Word(alphas)) + "END"
Since "END" matches the repetition expression "Word(alphas)", it will
never get parsed as the terminating sentinel. Up until now, this has
to be resolved by the user inserting their own negative lookahead:
"BEGIN" + OneOrMore(~Literal("END") + Word(alphas)) + "END"
Using stopOn, they can more easily write:
"BEGIN" + OneOrMore(Word(alphas), stopOn="END") + "END"
The stopOn argument can be a literal string or a pyparsing expression.
Inspired by a question by Lamakaha on StackOverflow (and many previous
questions with the same negative-lookahead resolution).
- Added expression names for many internal and builtin expressions, to
reduce name and error message overhead during parsing.
- Converted helper lambdas to functions to refactor and add docstring
support.
- Fixed ParseResults.asDict() to correctly convert nested ParseResults
values to dicts.
- Cleaned up some examples, fixed typo in fourFn.py identified by
aristotle2600 on reddit.
- Removed keepOriginalText helper method, which was deprecated ages ago.
Superceded by originalTextFor.
- Same for the Upcase class, which was long ago deprecated and replaced
with the upcaseTokens method.
Version 2.0.7 - December, 2015
------------------------------
- Simplified string representation of Forward class, to avoid memory
and performance errors while building ParseException messages. Thanks,
Will McGugan, Andrea Censi, and Martijn Vermaat for the bug reports and
test code.
- Cleaned up additional issues from enhancing the error messages for
Or and MatchFirst, handling Unicode values in expressions. Fixes Unicode
encoding issues in Python 2, thanks to Evan Hubinger for the bug report.
- Fixed implementation of dir() for ParseResults - was leaving out all the
defined methods and just adding the custom results names.
- Fixed bug in ignore() that was introduced in pyparsing 1.5.3, that would
not accept a string literal as the ignore expression.
- Added new example parseTabularData.py to illustrate parsing of data
formatted in columns, with detection of empty cells.
- Updated a number of examples to more current Python and pyparsing
forms.
Version 2.0.6 - November, 2015
------------------------------
- Fixed a bug in Each when multiple Optional elements are present.
Thanks for reporting this, whereswalden on SO.
- Fixed another bug in Each, when Optional elements have results names
or parse actions, reported by Max Rothman - thank you, Max!
- Added optional parseAll argument to runTests, whether tests should
require the entire input string to be parsed or not (similar to
parseAll argument to parseString). Plus a little neaten-up of the
output on Python 2 (no stray ()'s).
- Modified exception messages from MatchFirst and Or expressions. These
were formerly misleading as they would only give the first or longest
exception mismatch error message. Now the error message includes all
the alternatives that were possible matches. Originally proposed by
a pyparsing user, but I've lost the email thread - finally figured out
a fairly clean way to do this.
- Fixed a bug in Or, when a parse action on an alternative raises an
exception, other potentially matching alternatives were not always tried.
Reported by TheVeryOmni on the pyparsing wiki, thanks!
- Fixed a bug to dump() introduced in 2.0.4, where list values were shown
in duplicate.
Version 2.0.5 - October, 2015
-----------------------------
- (&$(@#&$(@!!!! Some "print" statements snuck into pyparsing v2.0.4,
breaking Python 3 compatibility! Fixed. Reported by jenshn, thanks!
Version 2.0.4 - October, 2015
-----------------------------
- Added ParserElement.addCondition, to simplify adding parse actions
that act primarily as filters. If the given condition evaluates False,
pyparsing will raise a ParseException. The condition should be a method
with the same method signature as a parse action, but should return a
boolean. Suggested by Victor Porton, nice idea Victor, thanks!
- Slight mod to srange to accept unicode literals for the input string,
such as "[а-яА-Я]" instead of "[\u0430-\u044f\u0410-\u042f]". Thanks
to Alexandr Suchkov for the patch!
- Enhanced implementation of replaceWith.
- Fixed enhanced ParseResults.dump() method when the results consists
only of an unnamed array of sub-structure results. Reported by Robin
Siebler, thanks for your patience and persistence, Robin!
- Fixed bug in fourFn.py example code, where pi and e were defined using
CaselessLiteral instead of CaselessKeyword. This was not a problem until
adding a new function 'exp', and the leading 'e' of 'exp' was accidentally
parsed as the mathematical constant 'e'. Nice catch, Tom Grydeland - thanks!
- Adopt new-fangled Python features, like decorators and ternary expressions,
per suggestions from Williamzjc - thanks William! (Oh yeah, I'm not
supporting Python 2.3 with this code any more...) Plus, some additional
code fixes/cleanup - thanks again!
- Added ParserElement.runTests, a little test bench for quickly running
an expression against a list of sample input strings. Basically, I got
tired of writing the same test code over and over, and finally added it
as a test point method on ParserElement.
- Added withClass helper method, a simplified version of withAttribute for
the common but annoying case when defining a filter on a div's class -
made difficult because 'class' is a Python reserved word.
Version 2.0.3 - October, 2014
-----------------------------
- Fixed escaping behavior in QuotedString. Formerly, only quotation
marks (or characters designated as quotation marks in the QuotedString
constructor) would be escaped. Now all escaped characters will be
escaped, and the escaping backslashes will be removed.
- Fixed regression in ParseResults.pop() - pop() was pretty much
broken after I added *improvements* in 2.0.2. Reported by Iain
Shelvington, thanks Iain!
- Fixed bug in And class when initializing using a generator.
- Enhanced ParseResults.dump() method to list out nested ParseResults that
are unnamed arrays of sub-structures.
- Fixed UnboundLocalError under Python 3.4 in oneOf method, reported
on Sourceforge by aldanor, thanks!
- Fixed bug in ParseResults __init__ method, when returning non-ParseResults
types from parse actions that implement __eq__. Raised during discussion
on the pyparsing wiki with cyrfer.
Version 2.0.2 - April, 2014
---------------------------
- Extended "expr(name)" shortcut (same as "expr.setResultsName(name)")
to accept "expr()" as a shortcut for "expr.copy()".
- Added "locatedExpr(expr)" helper, to decorate any returned tokens
with their location within the input string. Adds the results names
locn_start and locn_end to the output parse results.
- Added "pprint()" method to ParseResults, to simplify troubleshooting
and prettified output. Now instead of importing the pprint module
and then writing "pprint.pprint(result)", you can just write
"result.pprint()". This method also accepts addtional positional and
keyword arguments (such as indent, width, etc.), which get passed
through directly to the pprint method
(see https://docs.python.org/2/library/pprint.html#pprint.pprint).
- Removed deprecation warnings when using '<<' for Forward expression
assignment. '<<=' is still preferred, but '<<' will be retained
for cases where '<<=' operator is not suitable (such as in defining
lambda expressions).
- Expanded argument compatibility for classes and functions that
take list arguments, to now accept generators as well.
- Extended list-like behavior of ParseResults, adding support for
append and extend. NOTE: if you have existing applications using
these names as results names, you will have to access them using
dict-style syntax: res["append"] and res["extend"]
- ParseResults emulates the change in list vs. iterator semantics for
methods like keys(), values(), and items(). Under Python 2.x, these
methods will return lists, under Python 3.x, these methods will
return iterators.
- ParseResults now has a method haskeys() which returns True or False
depending on whether any results names have been defined. This simplifies
testing for the existence of results names under Python 3.x, which
returns keys() as an iterator, not a list.
- ParseResults now supports both list and dict semantics for pop().
If passed no argument or an integer argument, it will use list semantics
and pop tokens from the list of parsed tokens. If passed a non-integer
argument (most likely a string), it will use dict semantics and
pop the corresponding value from any defined results names. A
second default return value argument is supported, just as in
dict.pop().
- Fixed bug in markInputline, thanks for reporting this, Matt Grant!
- Cleaned up my unit test environment, now runs with Python 2.6 and
3.3.
Version 2.0.1 - July, 2013
--------------------------
- Removed use of "nonlocal" that prevented using this version of
pyparsing with Python 2.6 and 2.7. This will make it easier to
install for packages that depend on pyparsing, under Python
versions 2.6 and later. Those using older versions of Python
will have to manually install pyparsing 1.5.7.
- Fixed implementation of <<= operator to return self; reported by
Luc J. Bourhis, with patch fix by Mathias Mamsch - thanks, Luc
and Mathias!
Version 2.0.0 - November, 2012
------------------------------
- Rather than release another combined Python 2.x/3.x release
I've decided to start a new major version that is only
compatible with Python 3.x (and consequently Python 2.7 as
well due to backporting of key features). This version will
be the main development path from now on, with little follow-on
development on the 1.5.x path.
- Operator '<<' is now deprecated, in favor of operator '<<=' for
attaching parsing expressions to Forward() expressions. This is
being done to address precedence of operations problems with '<<'.
Operator '<<' will be removed in a future version of pyparsing.
Version 1.5.7 - November, 2012
-----------------------------
- NOTE: This is the last release of pyparsing that will try to
maintain compatibility with Python versions < 2.6. The next
release of pyparsing will be version 2.0.0, using new Python
syntax that will not be compatible for Python version 2.5 or
older.
- An awesome new example is included in this release, submitted
by Luca DellOlio, for parsing ANTLR grammar definitions, nice
work Luca!
- Fixed implementation of ParseResults.__str__ to use Pythonic
''.join() instead of repeated string concatenation. This
purportedly has been a performance issue under PyPy.
- Fixed bug in ParseResults.__dir__ under Python 3, reported by
Thomas Kluyver, thank you Thomas!
- Added ParserElement.inlineLiteralsUsing static method, to
override pyparsing's default behavior of converting string
literals to Literal instances, to use other classes (such
as Suppress or CaselessLiteral).
- Added new operator '<<=', which will eventually replace '<<' for
storing the contents of a Forward(). '<<=' does not have the same
operator precedence problems that '<<' does.
- 'operatorPrecedence' is being renamed 'infixNotation' as a better
description of what this helper function creates. 'operatorPrecedence'
is deprecated, and will be dropped entirely in a future release.
- Added optional arguments lpar and rpar to operatorPrecedence, so that
expressions that use it can override the default suppression of the
grouping characters.
- Added support for using single argument builtin functions as parse
actions. Now you can write 'expr.setParseAction(len)' and get back
the length of the list of matched tokens. Supported builtins are:
sum, len, sorted, reversed, list, tuple, set, any, all, min, and max.
A script demonstrating this feature is included in the examples
directory.
- Improved linking in generated docs, proposed on the pyparsing wiki
by techtonik, thanks!
- Fixed a bug in the definition of 'alphas', which was based on the
string.uppercase and string.lowercase "constants", which in fact
*aren't* constant, but vary with locale settings. This could make
parsers locale-sensitive in a subtle way. Thanks to Kef Schecter for
his diligence in following through on reporting and monitoring
this bugfix!
- Fixed a bug in the Py3 version of pyparsing, during exception
handling with packrat parsing enabled, reported by Catherine
Devlin - thanks Catherine!
- Fixed typo in ParseBaseException.__dir__, reported anonymously on
the SourceForge bug tracker, thank you Pyparsing User With No Name.
- Fixed bug in srange when using '\x###' hex character codes.
- Addeed optional 'intExpr' argument to countedArray, so that you
can define your own expression that will evaluate to an integer,
to be used as the count for the following elements. Allows you
to define a countedArray with the count given in hex, for example,
by defining intExpr as "Word(hexnums).setParseAction(int(t[0],16))".
Version 1.5.6 - June, 2011
----------------------------
- Cleanup of parse action normalizing code, to be more version-tolerant,
and robust in the face of future Python versions - much thanks to
Raymond Hettinger for this rewrite!
- Removal of exception cacheing, addressing a memory leak condition
in Python 3. Thanks to Michael Droettboom and the Cape Town PUG for
their analysis and work on this problem!
- Fixed bug when using packrat parsing, where a previously parsed
expression would duplicate subsequent tokens - reported by Frankie
Ribery on stackoverflow, thanks!