Using Flex and Bison with Qt

Flex and Bison, the first is a lexical analyzer generator, and the the second a parser generator, two useful tools for making programs that reacts to a data input with a predefined structure and language, for example, we can make compilers, script interpreters and command line parsers.
This is not aimed to be a tutorial of Flex and Bison, if you want to learn how to use this programs I encourage reading their respective manuals, but a simple example on how to integrate these to Qt.

Project file setup

First, we must add the libraries where are defined some symbols and functions of Flex and Bison.

LIBS += -lfl -ly

Later, we will create two custom variables in which we will add the path to their respective files, then you must add these variables to the OTHER_FILES variable to make it editable in Qt Creator.

FLEXSOURCES = lexer.l
BISONSOURCES = parser.y

OTHER_FILES +=  \
    $$FLEXSOURCES \
    $$BISONSOURCES

Flex and Bison must generate the source code of the lexer and the parser before our program's compilation, for that purpose we will use the QMAKE_EXTRA_COMPILERS variable, if you want to know more about these special variables you must read this wonderful article.

flexsource.input = FLEXSOURCES
flexsource.output = ${QMAKE_FILE_BASE}.cpp
flexsource.commands = flex --header-file=${QMAKE_FILE_BASE}.h -o ${QMAKE_FILE_BASE}.cpp ${QMAKE_FILE_IN}
flexsource.variable_out = SOURCES
flexsource.name = Flex Sources ${QMAKE_FILE_IN}
flexsource.CONFIG += target_predeps

QMAKE_EXTRA_COMPILERS += flexsource

flexheader.input = FLEXSOURCES
flexheader.output = ${QMAKE_FILE_BASE}.h
flexheader.commands = @true
flexheader.variable_out = HEADERS
flexheader.name = Flex Headers ${QMAKE_FILE_IN}
flexheader.CONFIG += target_predeps no_link

QMAKE_EXTRA_COMPILERS += flexheader

bisonsource.input = BISONSOURCES
bisonsource.output = ${QMAKE_FILE_BASE}.cpp
bisonsource.commands = bison -d --defines=${QMAKE_FILE_BASE}.h -o ${QMAKE_FILE_BASE}.cpp ${QMAKE_FILE_IN}
bisonsource.variable_out = SOURCES
bisonsource.name = Bison Sources ${QMAKE_FILE_IN}
bisonsource.CONFIG += target_predeps

QMAKE_EXTRA_COMPILERS += bisonsource

bisonheader.input = BISONSOURCES
bisonheader.output = ${QMAKE_FILE_BASE}.h
bisonheader.commands = @true
bisonheader.variable_out = HEADERS
bisonheader.name = Bison Headers ${QMAKE_FILE_IN}
bisonheader.CONFIG += target_predeps no_link

QMAKE_EXTRA_COMPILERS += bisonheader

Lexer setup

Here's a Flex file chunk with the most important explanation.

%{
// In this section we can add all needed headers, from Qt or another libraries.
#include <QtScript>

// Also, we must add the parser's header where are defined the tokens.
#include "parser.h"
%}

%%

 /* Parse intiger numbers */
-?[0-9]+ {
    // yyval is a variable created by Flex, which we can return the parsed
    // value.
    // yytext is another variable created by Flex that contains the text to
    // parse.
    yylval.QVariant_t = new QVariant();
    *yylval.QVariant_t = QString(yytext).toInt();

    return TOK_INTIGER;
}

%%

Parser setup

And here we have, a chunk of the parser.

%{
#include <QtGui>

// yylex is a function generated by Flex and we must tell to Bison that it is
// defined in other place.
extern int yylex(void);

// Bison uses the yyerror function for informing us when a parsing error has
// occurred.
void yyerror(const char *s);
%}

// Here we define our custom variable types.
// Custom types must be of fixed size.
%union {
    QVariant *QVariant_t;
}

// Define the terminal expression types.
%token <QVariant_t> TOK_INTIGER

// Define the non-terminal expression types.
%type <QVariant_t> variantListItems

%%

variantListItems: variant {
                      // $$ is a reference to variantListItems that is a
                      // QVariant.
                      $$ = new QVariant();

                      QVariantList variantList;

                      // $1, $2, $3, ..., $N are references to each expression.
                      // $1 is a QVariant.
                      variantList << *$1;

                      *$$ = variantList;
                  }
                | variantListItems TOK_COMMA variant {
                      $$ = new QVariant();

                      QVariantList variantList($1->toList());

                      variantList << *$3;

                      *$$ = variantList;
                  }
                ;

%%

void yyerror(const char *s)
{
    qDebug() << "error:" << s;
}

Add the parsers to our program

The last step is to add the headers and use the generated parsers.

#include <QtCore>

// Add the headers generated by Flex and Bison.
#include "lexer.h"
#include "parser.h"

int main(int argc, char *argv[])
{
    QCoreApplication app(argc, argv);

    // Let test the parser with this input.
    QString str("{'one': 1,"
                "\"two\": '2',"
                "\"'3'\": [3.14, 10, [7, 8], \"\\\"quoted string\\\"\"],"
                "'4': size(640, 480),"
                "'date': dateTime(date(2013, 2, 27), time(12, 0)),"
                "'number count': {'1': 1, '2': 2, '3': 3}}");

    // Bison take the standard input (command line) as input.
    // yy_scan_string will switch the input to a string.
    YY_BUFFER_STATE bufferState = yy_scan_string(str.toUtf8().constData());

    // Parse the string.
    yyparse();

    // and release the buffer.
    yy_delete_buffer(bufferState);

    return app.exec();
}

Finally, the output will be some thing like this:

QVariant
(
    QVariantMap, QMap
    (
        ( "one" ,  QVariant(int, 1) )
        ( "two" ,  QVariant(QString, "2") )
        (
            "'3'", QVariant
            (
                QVariantList,
                (
                    QVariant(float, 3.14) , 
                    QVariant(int, 10) , 
                    QVariant
                    (
                        QVariantList, (QVariant(int, 7) ,  QVariant(int, 8) ) 
                    ) , 
                    QVariant(QString, ""quoted string"")
                ) 
            )
        )
        ( "4" ,  QVariant(QSize, QSize(640, 480) ) )
        (
            "date" ,  QVariant
            (
                QDateTime, QDateTime("mié feb 27 12:00:00 2013")
            )
        )
        (
            "number count" ,  QVariant
            (
                QVariantMap, QMap
                (
                    ( "1", QVariant(int, 1) )
                    ( "2", QVariant(int, 2) )
                    ( "3", QVariant(int, 3) )
                ) 
            )
        )
    ) 
)

You can download the full source code of the example from my github.
PD: BTW, if you paid attention to the source code, there are a memory leaks in the program, I will left you the job of solving it ;)