This file contains an SQL parser based on lex/yacc that was posted to comp.databases in 1993, after having been originally posted in 1989. From: dtb@otto.bf.rmit.oz.au (David Bath) Newsgroups: comp.databases Subject: SQL parser lex/yacc source Keywords: SQL lex yacc Date: 11 May 93 06:35:47 GMT Organization: Royal Melbourne Institute of Technology It's been a while, but I finally managed to get a non-corrupt copy of the lex/yacc for a SQL parser onto this machine. It was written by Leroy Cain (who I cannot find - but all credit to him) It makes OK in a System V.3 provided you tweak the makefile, changing "yf" to "./yf" (or include . in your PATH - uuuuuuugh!). In BSD based systems I have had some problems - I had problems with the sed commands - they were garbled (so I used the AT&T sed) and the final link had one unresolved symbol. I probably could have got it to completion with a bit more work, but hey, I am mainly Sys V anyway. Path: iam!chx400!cernvax!mcsun!uunet!cuc1!lcain From: lcain@cuc1.UUCP (Leroy Cain) Newsgroups: comp.databases Subject: Re: Parser for SQL? Summary: ISO/ANSI SQL parser Keywords: SQL, yacc Date: 3 Nov 89 15:04:12 GMT Organization: Columbia Union College; Takoma Park, MD 20912 In article <1989Oct30.121344.2112@iam.unibe.ch>, metz@iam.unibe.ch (Igor Metz) writes: > > Hi, > I'm probably not the first to ask about a yacc source for SQL, but I'm new to > this newsgroup. > Does somebody have a yacc parser for SQL which she/he would share with us? > Thanks in advance for your help. > -- > Igor Metz X400: metz@iam.unibe.ch > Institut fuer Informatik ARPA: metz%iam.unibe.ch@relay.cs.net > und angewandte Mathematik UUCP: ..!uunet!mcvax!iam.unibe.ch!metz > Universitaet Bern Phone: (0041) 31 65 49 90 The parser is billed as a frame work from which to build SQL query processing componet. The basic idea is to drop C code into the LEX and YACC files and extend the existing C files. To specificly answer question: 1. Testing. There are three executables ansi_sql, esql and module. Each one corrisponds to the three componets of the system. The function level function call interface which is useful for talk with a database engine is test by ansi_sql; The Embedded SQL is test by esql; and the Module interface is test by module. When you run ansi_sql you should get a "1>>" this is prompting for a SQL command. Try "SELECT * FROM Junk;" it should respond with Reserved Words 'SELECT' Reserved Words 'FROM' Got an IDENTIFIER 'Junk' 1>> exit using control d. Esql can be passed a file "esql file.esql" file.esql: main() { EXEC SQL BEGIN DECLARE SECTION END EXEC CHAR Tvar1[10]; INT Tvar2; CHAR Tvar3[10]; FLOAT Tvar4; EXEC SQL END DECLARE SECTION END EXEC } Module can be pass a file "module file.mod" file.mod: MODULE Test_001 LANGUAGE C AUTHORIZATION Lcain PROCEDURE Close_cur SQLCODE; CLOSE Cur1; PROCEDURE Commit_tran SQLCODE; COMMIT WORK; PROCEDURE Delete_p SQLCODE; DELETE FROM Tabname WHERE CURRENT OF Curname; PROCEDURE Delete_s SQLCODE; DELETE FROM Tabname WHERE Col1 = +5.0; PROCEDURE Fetch_ SQLCODE; FETCH Curname INTO Col1 INDICATOR Colind1; PROCEDURE Insert_ SQLCODE; INSERT INTO Tabname (Col1,Col2,Col3) VALUES ( 'Test',+5,-5.8 ); PROCEDURE Open_cur SQLCODE; OPEN Cur1; PROCEDURE Rollback_ SQLCODE; ROLLBACK WORK; PROCEDURE List_table Colname CHARACTER( 10) SQLCODE; SELECT Colname INTO Colname INDICATOR Colind FROM Tabname; PROCEDURE Update_p SQLCODE; UPDATE Tabname SET Col1 = Col1 * +3.5 WHERE CURRENT OF Curname; PROCEDURE Update_s SQLCODE; UPDATE Tabname SET Col2 = Col2 * +3.5 WHERE Col1 = 'Test'; 2. Integerating into your product. Since I don't know how your product is structure I will give you a general answer. If you need more specific information I will need to know more about you product. See the following documents for information on the Standard: ANSI X3.135-1986 and ANSI x3.135.1-198x ISO 9075-1987(E) Addendum-1 By adding the appropriate C code to the asql.y file one can create parsers that are usable with the function interface, ESQL and Module componets. At least this is the theory. Only common code should go in there. Any code specific to each componet ( other than when inclose in the @ @ condition YACCalation) should be put in a separate file. The function level interface can be tied to a database engine. Note that the function level interface is incomplete and pretty shitty. So for now you have to invent your own, Sorry about that. The ESQL takes a file in; filters the ESQL statements; and substitutes function call of your choice. You may want to tie it to the database to get info on fields and tables for sematic processing. The Module interface works about the same as ESQL except there is no high level language in module files. NOTE: I found a bug int test.c Add at Line 50 of test if(c==EOF) return c; Thanks for you input. Leroy Cain; Columbia Union College; Mathematical Sciences Department 7600 Flower Ave. WH406; Takoma Park, Md 20912 (301) 891-4172 uunet!cucstud!lcain I've been holding off releasing my parser in hopes of a better product. Since I haven't received any bug reports recently from my volunteer testers I am assume that it is working adequately. So here it is. So far I don't have any good example files so if some one comes up with a good set I would like a copy. I would appreciate any comments or suggestions. Leroy Cain; Columbia Union College; Mathematical Sciences Department 7600 Flower Ave. WH406; Takoma Park, Md 20912 (301) 891-4172 uunet!cucstud!lcain -----Cut Here-----Cut Here-----Cut Here-----Cut Here----- #!/bin/sh # shar: Shell Archiver # Run the following text with /bin/sh to create: # Makefile # README # asql.l # asql.y # asqlca.h # asqlda.h # test.c # module.c # esqlc.c # sqlfunc.c # yf.c sed 's/^X//' << 'SHAR_EOF' > Makefile X# X# This software is Copyright (c) 1989 by Leroy Cain. X# X# Permission is hereby granted to copy, reproduce, redistribute or X# otherwise use this software as long as: there is no monetary X# profit gained specifically from the use or reproduction or this X# software, it is not sold, rented, traded or otherwise marketed X# except under licence from the author, and this copyright notice X# is included prominently in any copy made. X# X# The author makes no claims as to the fitness or correctness of X# this software for any use whatsoever, and it is provided as is. X# Any use of this software is at the user's own risk. X# X# X#ESQL Languages X# C X# COBOL X# FORTRAN X# PASCAL X# PLI XLANG=C X X# if you have ansi c X#AFLAG= -Dansi X Xall: ansi_sql esql module X X############################################################################## X# ansi_sql is an interactive version of SQL X Xansi_sql: test.o libsql.a X cc -o ansi_sql test.o libsql.a X Xtest.o: test.c X cc -c $(AFLAG) test.c X X############################################################################## X# This section produces a library of function that is called by ESQL X# and MODULE programs. X Xlibsql.a: y.inter.o sqlfunc.o X ar r libsql.a y.inter.o sqlfunc.o X# ranlib libsql.a X Xy.inter.o: y.inter.c ilex.yy.c X cc -c -DINTERNAL $(AFLAG) y.inter.c X Xy.inter.c: asql.y yf X yf INTERNAL inter.y X yacc -v inter.y X mv y.tab.c y.inter.c X X############################################################################## X Xesql: y.esql.o esqlc.o X cc -o esql esqlc.o y.esql.o X Xesqlc.o:esqlc.c X cc -c $(AFLAG) esqlc.c X Xy.esql.o: y.esql.c lex.yy.c X cc -c -DEMBEDDED y.esql.c X Xy.esql.c: asql.y yf X yf EMBEDDED esql.y X yacc -vd esql.y X mv y.tab.c y.esql.c X mv y.tab.h y.esql.h X X############################################################################## X Xmodule: module.o y.mod.o X cc -o module module.o y.mod.o X Xmodule.o:module.c X cc -c $(AFLAG) module.c X Xy.mod.o: y.mod.c lex.yy.c X cc -c -DMODULE y.mod.c X Xy.mod.c: asql.y yf X yf MODULE mod.y X yacc -v mod.y X mv y.tab.c y.mod.c X X############################################################################## X Xlex.yy.c: asql.l X lex asql.l X Xilex.yy.c: asql.l X lex asql.l X sed -e "s/getc(yyin)/(*sqlbuf++)/1" lex.yy.c >ilex.yy.c X X############################################################################## X# yf is a filter program to separate up asql.y into to three componets X Xyf:yf.c X cc -o yf yf.c X X############################################################################## X# clean things up Xclean: X rm *.o esql.y inter.y mod.y ilex.yy.c lex.yy.c libsql.a \ X y.esql.c y.mod.c y.inter.c SHAR_EOF sed 's/^X//' << 'SHAR_EOF' > README XLeroy Cain; Columbia Union College; Mathematical Sciences Department X7600 Flower Ave. WH406; Takoma Park, Md 20912 X(301) 891-4172 uunet!cucstud!lcain X XThis is an parser for ISO/ANSI SQL written useing LEX, YACC, and C. XIt provides syntax check and a frame for building a completed query processor. XSee ANSI X3.135-1986 and ANSI X3.135.1-198x Addendum 1 X ISO 9075-1987(E) Addendum 1 X XUnlike the previous SQL parser I posted this one can be process with Xa normal YACC so everyone should be able to use it. X XThere are three parts: X 1. Module -- This process the module specification and X produces function calls to which ever language X is specificied. X 2. Embedded -- This process the embedded SQL specification and X produces function calls to which ever language X is specificied. X 3. Internal -- This process SQL command passed to the function X interface. X XNotes: X In Module and Embedded you add the code to produce the function X calls. X Language specificied for use with Module and Embedded C, COBOL, X FORTRAN, Pascal, PL/1. X C as a possible langauge for Module and Embedded is not specificied X in the ANSI specs. This is the only place I deveated. X XHow to use this package X Running make will produce the executables module, esql, ansi_sql. SHAR_EOF sed 's/^X//' << 'SHAR_EOF' > asql.l X%{ X /* X * This software is Copyright (c) 1989 by Leroy Cain. X * X * Permission is hereby granted to copy, reproduce, redistribute or X * otherwise use this software as long as: there is no monetary X * profit gained specifically from the use or reproduction or this X * software, it is not sold, rented, traded or otherwise marketed X * except under licence from the author, and this copyright notice X * is included prominently in any copy made. X * X * The author makes no claims as to the fitness or correctness of X * this software for any use whatsoever, and it is provided as is. X * Any use of this software is at the user's own risk. X * X */ X%} X/* 5.1 */ XDIGIT [0-9] XLOWER [a-z] XUPPER [A-Z] XLETTER ({UPPER}|{LOWER}) XSPECIAL [!-/] XEOL [\n\r] XCHARACTER {DIGIT}|{LETTER}|{SPECIAL} X X/* 5.2 */ XUINT {DIGIT}+ XSINT [+-]{DIGIT}+ XENL ([+-])?({UINT}"."{UINT})|({UINT}".")|("."{UINT}) XANL {ENL}[Ee]{SINT} X X%% X{UPPER}("_"|{LETTER}|{DIGIT})* X{ Xint rw; X/* 5.3 */ X if(rw=rwlookup(yytext)) X { X printf("Reserved Words '%s'\n",yytext); X return rw; X } X else X { X printf("Got an IDENTIFIER '%s'\n",yytext); X return IDENTIFIER; X } X} X X({ENL})|({ANL}) X{ X return NUMERIC; X} X X{SINT} X{ X return SINTEGER; X} X X{UINT} X{ X return UINTEGER; X} X X\'[^\']*\' X{ X return STRING; X} X"," return COMA; X"(" return LPAREN; X")" return RPAREN; X"[" return LBRACKET; X"]" return RBRACKET; X"<" return GT; X">" return LT; X"." return PERIOD; X":" return COLON; X";" return SEMICOLON; X"=" return EQ; X"*" return MULT; X"+" return PLUS; X"-" return SUB; X"/" return DIV; X"<>" return NEQ; X">=" return GE; X"<=" return LE; X--[^\n]*\n /* COMMENT */; X[ \n\t] /* SPACE */; X. return ERROR; X%% X Xstatic struct X{ X char *word; X int token; X} rwords[] = X{ X"ALL", RW_ALL, X"AND", RW_AND, X"ANY", RW_ANY, X"AS", RW_AS, X"ASC", RW_ASC, X"AUTHORIZATION", RW_AUTHORIZATION, X"AVG", RW_AVG, X"BEGIN", RW_BEGIN, X"BETWEEN", RW_BETWEEN, X"BY", RW_BY, X"C", RW_C, X"CHAR", RW_CHAR, X"CHARACTER", RW_CHARACTER, X"CHECK", RW_CHECK, X"CLOSE", RW_CLOSE, X"COBOL", RW_COBOL, X"COMMIT", RW_COMMIT, X"CONTINUE", RW_CONTINUE, X"COUNT", RW_COUNT, X"CREATE", RW_CREATE, X"CURRENT", RW_CURRENT, X"CURSOR", RW_CURSOR, X"DEC", RW_DECIMAL, X"DECIMAL", RW_DECIMAL, X"DECLARE", RW_DECLARE, X"DEFAULT", RW_DEFAULT, X"DELETE", RW_DELETE, X"DESC", RW_DESC, X"DISTINCT", RW_DISTINCT, X"DOUBLE", RW_DOUBLE, X"END", RW_END, X"ESCAPE", RW_ESCAPE, X"EXEC", RW_EXEC, X"EXISTS", RW_EXISTS, X"FETCH", RW_FETCH, X"FLOAT", RW_FLOAT, X"FOR", RW_FOR, X"FOREIGN", RW_FOREIGN, X"FORTRAN", RW_FORTRAN, X"FOUND", RW_FOUND, X"FROM", RW_FROM, X"GO", RW_GO, X"GOTO", RW_GOTO, X"GRANT", RW_GRANT, X"GROUP", RW_GROUP, X"HAVING", RW_HAVING, X"IN", RW_IN, X"INDICATOR", RW_INDICATOR, X"INSERT", RW_INSERT, X"INT", RW_INT, X"INTEGER", RW_INTEGER, X"INTO", RW_INTO, X"IS", RW_IS, X"KEY", RW_KEY, X"LANGUAGE", RW_LANGUAGE, X"LIKE", RW_LIKE, X"MAX", RW_MAX, X"MIN", RW_MIN, X"MODULE", RW_MODULE, X"NOT", RW_NOT, X"NULL", RW_NULL, X"NUMERIC", RW_NUMERIC, X"OF", RW_OF, X"ON", RW_ON, X"OPEN", RW_OPEN, X"OPTION", RW_OPTION, X"OR", RW_OR, X"ORDER", RW_ORDER, X"PASCAL", RW_PASCAL, X"PLI", RW_PLI, X"PRECISION", RW_PRECISION, X"PRIMARY", RW_PRIMARY, X"PRIVILEGES", RW_PRIVILEGES, X"PROCEDURE", RW_PROCEDURE, X"PUBLIC", RW_PUBLIC, X"REAL", RW_REAL, X"REFERENCES", RW_REFERENCES, X"ROLLBACK", RW_ROLLBACK, X"SCHEMA", RW_SCHEMA, X"SECTION", RW_SECTION, X"SELECT", RW_SELECT, X"SET", RW_SET, X"SMALLINT", RW_SMALLINT, X"SOME", RW_SOME, X"SQL", RW_SQL, X"SQLCODE", RW_SQLCODE, X"SQLERROR", RW_SQLERROR, X"SUM", RW_SUM, X"TABLE", RW_TABLE, X"TO", RW_TO, X"UNION", RW_UNION, X"UNIQUE", RW_UNIQUE, X"UPDATE", RW_UPDATE, X"USER", RW_USER, X"VALUES", RW_VALUES, X"VIEW", RW_VIEW, X"WHENEVER", RW_WHENEVER, X"WHERE", RW_WHERE, X"WITH", RW_WITH, X"WORK", RW_WORK, X"",0 X}; X Xrwlookup(rw) Xchar *rw; X{ Xint x; X X for(x=0;rwords[x].token && strcmp(rw,rwords[x].word);x++); X return rwords[x].token; X} SHAR_EOF sed 's/^X//' << 'SHAR_EOF' > asql.y X/* X * This software is Copyright (c) 1989 by Leroy Cain. X * X * Permission is hereby granted to copy, reproduce, redistribute or X * otherwise use this software as long as: there is no monetary X * profit gained specifically from the use or reproduction or this X * software, it is not sold, rented, traded or otherwise marketed X * except under licence from the author, and this copyright notice X * is included prominently in any copy made. X * X * The author makes no claims as to the fitness or correctness of X * this software for any use whatsoever, and it is provided as is. X * Any use of this software is at the user's own risk. X * X */ X%token IDENTIFIER X%token UINTEGER X%token SINTEGER X%token NUMERIC X%token STRING X X%token COMA X%token LPAREN X%token RPAREN X%token LBRACKET X%token RBRACKET X%token PERIOD X%token COLON X%token SEMICOLON X%right NEG X%left DIV MULT X%left PLUS SUB X%left EQ NEQ GT LT GE LE X%token ERROR X X%token RW_CHAR X RW_INT X X%token RW_ALL X RW_AND X RW_ANY X RW_AS X RW_ASC X RW_AUTHORIZATION X RW_AVG X RW_BEGIN X RW_BETWEEN X RW_BY X RW_C X RW_CHARACTER X RW_CHECK X RW_CLOSE X RW_COBOL X RW_COMMIT X RW_CONTINUE X RW_COUNT X RW_CREATE X RW_CURRENT X RW_CURSOR X RW_DECIMAL X RW_DECLARE X RW_DEFAULT X RW_DELETE X RW_DESC X RW_DISTINCT X RW_DOUBLE X RW_END X RW_ESCAPE X RW_EXEC X RW_EXISTS X RW_FETCH X RW_FLOAT X RW_FOR X RW_FOREIGN X RW_FORTRAN X RW_FOUND X RW_FROM X RW_GO X RW_GOTO X RW_GRANT X RW_GROUP X RW_HAVING X RW_IN X RW_INDICATOR X RW_INSERT X RW_INTEGER X RW_INTO X RW_IS X RW_KEY X RW_LANGUAGE X RW_LIKE X RW_MAX X RW_MIN X RW_MODULE X RW_NOT X RW_NULL X RW_NUMERIC X RW_OF X RW_ON X RW_OPEN X RW_OPTION X RW_OR X RW_ORDER X RW_PASCAL X RW_PLI X RW_PRECISION X RW_PRIMARY X RW_PRIVILEGES X RW_PROCEDURE X RW_PUBLIC X RW_REAL X RW_REFERENCES X RW_ROLLBACK X RW_SCHEMA X RW_SECTION X RW_SELECT X RW_SET X RW_SMALLINT X RW_SOME X RW_SQL X RW_SQLCODE X RW_SQLERROR X RW_SUM X RW_TABLE X RW_TO X RW_UNION X RW_UNIQUE X RW_UPDATE X RW_USER X RW_VALUES X RW_VIEW X RW_WHENEVER X RW_WHERE X RW_WITH X RW_WORK X X%% X/*****************************************************************************/ X X/*----------------------------------------------------------------------------- XParser for Module X-----------------------------------------------------------------------------*/ X X@MODULE Xansi_sql_module: X module X ; X@ X X/*----------------------------------------------------------------------------- XParser for ESQL X-----------------------------------------------------------------------------*/ X X@EMBEDDED Xansi_sql_embedded: X embedded_sql_statement X { return ; } X | embedded_sql_declare_section X { return ; } X ; X@ X X/*----------------------------------------------------------------------------- XParser for SQL Statements X-----------------------------------------------------------------------------*/ X@INTERNAL Xansi_sql_internal_statement: X sql_statement X | declare_cursor X | schema X ; X@ X X/*------------------ B ---------------------*/ X X@EMBEDDED Xembedded_exception_declaration: X RW_WHENEVER condition exception_action X ; X Xcondition: X RW_SQLERROR X | RW_NOT RW_FOUND X ; X Xexception_action: X RW_CONTINUE X | RW_GOTO target X ; X Xtarget: X COLON host_identifier X | NUMERIC X ; X@ X X/*-------------------- A ------------------------*/ X X@EMBEDDED X Xembedded_sql_statement: X sql_prefix declare_cursor sql_terminator X | embedded_exception_declaration X | sql_prefix sql_statement sql_terminator X | sql_prefix schema sql_terminator X ; X Xsql_prefix: X RW_SQL X ; X Xsql_terminator: X RW_END RW_EXEC X | SEMICOLON X ; X Xembedded_sql_declare_section: X sql_prefix RW_BEGIN RW_DECLARE RW_SECTION sql_terminator X host_variable_definition_list SEMICOLON X RW_EXEC sql_prefix RW_END RW_DECLARE RW_SECTION sql_terminator X ; X Xhost_variable_definition_list: X c_variable_definition X | host_variable_definition_list SEMICOLON c_variable_definition X ; X Xc_variable_definition: X RW_CHAR IDENTIFIER X | RW_CHAR IDENTIFIER LBRACKET UINTEGER RBRACKET X | RW_INT IDENTIFIER X | RW_FLOAT IDENTIFIER X | RW_DOUBLE IDENTIFIER X ; X@ X X/*------------------ 8.12 ----------------------*/ X Xupdate_statement_search: X RW_UPDATE table_name X RW_SET set_clause_list X RW_WHERE search_condition X ; X X/*----------------- 8.11 ---------------------*/ X Xupdate_statement_position: X RW_UPDATE table_name X RW_SET set_clause_list X RW_WHERE RW_CURRENT RW_OF cursor_name X ; X Xset_clause_list: X set_clause X | set_clause_list COMA set_clause X ; X Xset_clause: X column_name EQ value_expression X | column_name EQ RW_NULL X ; X X/*----------------------- 8.10