14 May 2009, 15:28
Generic-user-small

Gobeaux Sebastien (5 posts)

I’m trying to build a programm that will check if PL/SQL’s input are conform with my companies own rules regarding Pl/SQL scripts. If not, the program have to indicate which expression (at which line) didn’t pass the test.

I’ve decided to use the PL/SQL grammar available on the ANTLR website. This grammar will check if there is syntaxic error. Then, i will apply different Tree grammar to check on the inner rules (company’s own rule).

But to do that, I need information regarding token! So here come the question :

How to keep information (such as position line) regarding tokens parsed in a grammar parser to a tree parser ?

Thanks,

14 May 2009, 14:14
Generic-user-small

Gobeaux Sebastien (5 posts)

ANTLR has been well designed ;)

So here is a solution for those who might have the same question.

The goal here is to check that the SELECT expression respect the following format:

SELECT , , ... FROM , ...

My first grammar doesn’t know about prefixing my column by the table name. So I have to implement the control during the tree grammar.

I began by adding to a list of Object both Column_name and table_name nodes. Then I check that the column_name start by one of the table_name.

Here the key command to get the line and position :

((CommonTree)C).getToken().getLine() ((CommonTree)C).getToken().getCharPositionInLine()


Grammar :

grammar jDemo_1_1g;

options {	
	output=AST;
	k=*;
	backtrack=true;
	memoize=true;
}
tokens {
  SELECT;   // variable SELECT
  FROM;
}


rules 	:	keySELECT sql_champ (',' sql_champ)* keyFROM sql_table (',' sql_table)*
 -> ^(SELECT sql_champ* ^(FROM sql_table*) )
	;

sql_champ 
	:	sql_expression
	;
	
sql_table
	:	sql_expression
	;

sql_expression 
	:	ID
	;

keySELECT: {(this.input).LT(1).getText().toUpperCase().equalsIgnoreCase("SELECT")}? ID;
keyFROM: {(this.input).LT(1).getText().toUpperCase().equalsIgnoreCase("FROM")}? ID;

ID :('a'..'z'|'A'..'Z'|'_')+ ;
WS :(' '|'\t'|'\n'|'\r'| 'ok') {$channel=HIDDEN;};

TreeGrammar

tree grammar jDemoWalker_1_0c;

options {
 tokenVocab=jDemo1_0; // Import tokens from CMinus.g
 ASTLabelType = CommonTree;
}
/*@members	{

}

scope {

}*/
rules 	
scope {
List table;
List champ
}
@init	{
//Two list of CommonTree node
$rules::table = new ArrayList();
$rules::champ = new ArrayList();
}
@after	{
    
         Boolean verif_global = true;
         Boolean verif_champ = true;
         
         String schamp; // field | column_name
         String stable;
                  
         Loop1: for (Object C : $rules::champ) {
            schamp = C.toString();

            Loop2: for (Object T : $rules::table) {
                 verif_champ = false;
                 stable = T.toString() + "_";
                 
                 if (schamp.indexOf(stable)!=-1) {
                     System.out.println(schamp+" correctly prefixed");
                     verif_champ = true;
                     break Loop2;
                 }
             }
             if (!verif_champ)   {
                 System.out.println(" Wrong or missing prefixes at line :" +((CommonTree)C).getToken().getLine() + "; position : " + *((CommonTree)C).getToken().getCharPositionInLine()*);
                 break boucle1;
             }
         }         
}
:	^(SELECT sql_champ* ^(FROM sql_table*) )
	// Afficher la liste des champs :
	{System.out.println("ola :" + $rules::champ);}
	;
	
sql_champ 
	:	sql_expression {$rules::champ.add($sql_expression.start);}
	;
	
sql_table
	:	sql_expression {$rules::table.add($sql_expression.start);}
	;

sql_expression 
	:	ID
	;
 
keySELECT: {(this.input).LT(1).toString().toUpperCase().equalsIgnoreCase("SELECT")}? ID;
keyFROM: {(this.input).LT(1).toString().toUpperCase().equalsIgnoreCase("FROM")}? ID;

Java main

public static void main(String[] args) throws IOException, RecognitionException {
       // Create an input character stream from standard in
        ANTLRInputStream input = new ANTLRInputStream(System.in);
        // Create an ExprLexer that feeds from that stream
        jDemo_1_0Lexer lexer = new jDemo_1_0Lexer(input);

        // Create a stream of tokens fed by the lexer
        CommonTokenStream tokens = new CommonTokenStream(lexer);
        // Create a parser that feeds off the token stream
        jDemo_1_0Parser parser = new jDemo_1_0Parser(tokens);
        jDemo_1_0Parser.rules_return r = parser.rules();
        CommonTree t = (CommonTree)r.getTree();
        System.out.println("Tree :" + t.toStringTree());
        // Begin parsing at rule prog
        /*parser.rules();*/
        
        CommonTreeNodeStream nodes = new CommonTreeNodeStream(t);
        nodes.setTokenStream(tokens);
        jDemoWalker_1_0c walker = new jDemoWalker_1_0c(nodes);
        walker.rules();
        
        return;
}

For this Input …

SELECT
dede_champ,
dudu_champ, dudu_champB, dada_champ
FROM dede, dudu
^Z

… output should be like this

Tree:(SELECT dede_champ dudu_champ dudu_champB dada_champ (FROM dede dudu))
ola :[dede_champ, dudu_champ, dudu_champB, dada_champ]
dede_champ  correctly prefixed
dudu_champ  correctly prefixed
dudu_champB  correctly prefixed
Wrong or missing prefixes at line :3; position : 25
02 Sep 2010, 08:50
Generic-user-small

reale eidos (1 post)

Any sample code for ANTLR 3 ?? thanks

  You must be logged in to comment