ttlogo.jpg Free TextTransformer Projects
Home
Text2HTML
Wikipedia
Yacc2TT
Delphi parser
Delphi preprocessor
Delphi pretty printer
Java parser
C preprocessor
C parser
HTML4
Utilities
MIME parser
Spamfilter
Additional Examples
Free components
  Minimal Website   Impressum

DELPHI

The beginnings of the Delphi.ttp project for the development of a parser for the programming language Delphi go back to the beginnings of the development of TextTransformer. The real aim was to create a translator to C++. Such a translator could make it easier for a C++ developer to adapt Delphi components to his programs. In the end, also TextTransformer would benefit from such a translator, since much such components are used in it.

The greatest difficulty at the development of a Delphi parser is, that there isn't any official standard for this programming language. The specification of the language was primarily taken from the help of the Delphi development environment. The information is scattered there fragmentarily, remains altogether incomplete and non-uniform. Moreover, Delphi permanently was developed further.
In the meantime, there is an excellent attempt to represent the Delphi specification: the "reference guide" to "Free Pascal".

http://www.freepascal.org/docs-html/ref/ref.html

There still are discrepancies in details. The parser project nevertheless was revised with this reference again, particularly the namings were adapted as far as possible. However, there is no identity, not at least, because the rules in the TextTransformer parser were made for efficiency reasons LL(1), if possible.


The parser presented here might approximately represent the state of Delphi 5. It is able to parse the complete VCL which is part of the CBuilder 6. It presupposes that the code to be parsed is syntactically correct. It cannot be used to check the correctness, because it also allows constructions which the Delphi compiler would reject.

Dr. Hans-Peter Diettrich has supported me at the development of the parser for a while; therefore my thanks!


Project options

The project options for the parser/scanner differ from the standard settings in three points:

  • The project DelphiPreproc.ttp is used as a preprocessor in front of the parser.
  • The recognition isn't case sensitive
  • Only the actual expected literals are tested

The use of the preprocessor permits to test the parser directly at the VCL.
The second point takes into account simply the language definition of Delphi.
The last point is necessary, because in Delphi also keywords like "index" or "end" may be used as names for variables and parameters. The disadvantage of this option is that unexpected keywords can be recognized as identifiers, what makes it more difficult, to find errors in the parser.


Comments


Ther are three styles for comments in Delphi:

  • the old Pascal style: "(*"... *)"
  • the style of Turbo Pascal: "{" ... "}"
  • as a line comment: // ...

In Delphi.ttp, these three forms are combined into one "COMMENT" production, which is set in the project options as an inclusion. A nesting of the comments wasn't allowed because there are cases in the VCL in which appropriately required closing brackets are missing. However, the occurrence of one kind of comment within another one is permitted. (The latter isn't the case for the alternatively usable regular expression "IGNORE", which was left in the project but not used.)


Warnings because of optional semicolons

When the project is compiled, there are some warnings of the kind, that certain tokens are start and successor of deletable structures. They result from the treatment of the semicolon. Semicolons divide both whole structures and sub-structures in Delphi, however, aren't always a necessary part of these structures. In this TextTransformer project the difficult decision, when the semicolons is necessary and when not are fairly often avoided by assuming their occurrence as optional; either as isolated optional tokens or as an alternative in repetitions. Since the parser presupposes that the code is correct, necessary semicolons are always recognized by the optional expressions. The token after the semicolon then must be able to decide on the further course of parsing. However, these nullable optional semicolons are causing mentioned warnings.

An example of a correct but complicated treatment of the semicolon is the production "stmt_list". This production specifies a list of instructions. In accordance with the method above it simply could be formulated in the following way:

stmt_list ::= ( stmt | ";" )+

All tokens, by which an instruction "stmt" can start and end are causing warnings then.

To avoid these warnings it can be tested with the look-ahead "is_end_of_stmt_list" whether a token follows the semicolon, by which lists of instructions are finished. A semicolon is not needed before one of the key words "end", "except", "finalization", "finally", and "until".

stmt_list ::=  ";" stmt_list_not_empty? | stmt_list_not_empty 
stmt_list_not_empty ::=  stmt ( IF(!is_end_of_stmt_list()) ";" stmt ELSE ";" END )* 

Other Delphi grammars

There are some attempts to represent the Delphi grammar, which haven't prospered far. But the following seems worth mentioning:

http://www.felix-colibri.com/papers/compilers/delphi_5_grammar/delphi_5_grammar.html

However, this grammar is written for a parser generator "GenCot", apparently of their own and in some cases additions of handwritten code is needed, which isn't published.

There are more grammars at:

http://www.geocities.com/robertzierer/Grammar.html

Tests

It was mentioned already above that the parser can parse the complete VCL belonging to the CBuilder 6. In addition it passes successfully a test suite, which was derived from from the Free Pascal tests.

http://www.freepascal.org/

The following changes were carried out at the original test suite:

  • Tests of Free Pascal specific expansions were removed. This concerns the generic types, the overloading of operators and the global properties.
  • Tests for the Macintosh Pascal variant were removed.
  • Source files were adapted where other typical Free Pascal properties are used. The original versions are still included in the zip-file with the file extension ".pp_org". Some changes were also necessary because the preprocessor is still insufficient.
  • The additional constants "CPUI386" and "fpc_fullversion" were defined in the preprocessor DelphiPreproc.ttp.
  • Delphi.ttp was extended by some Free Pascal keywords. They shall be removed later.
  • Only the positive tests weren't taken into account till now, but not the tests at which errors are expected.
Download of the test suite

Last update: 12/31/09
1.1.6 local options of the comment productions corrected, such that no characters are ignored any more
11/17/09 1.1.5
1.1.0 : passes successfully a test suite.
1.0.2 : A fault in the regular expression of CTRL_CHAR had hidden a conflict. The grammar for types was, revised. Now the conflict is avoided and in addition a look-ahead is removed.





 to the top