Contents

Previous

Next



Adding Parsers

Each parser in Source-Navigator is capable of understanding source files written in a specific programming language. In the Source-Navigator suite, parsers are stand-alone executables which adhere to a consistent command line interface. This interface allows Source-Navigator to control the parser's behavior through command line switches that the parser is expected to observe.

The Source-Navigator Software Development Kit (SDK) provides a C-based application programming interface (API) which enables parsers to insert information into a project database.

All files listed in SDK-Related Files can be found in the .../share/sdk directory.

SDK-Related Files

Subdirectory

Files

Description

/include/

sn.h
snptools.h

Include files

/lib/

libdbutils.a
libdb.a
libsnptools.a
libtcl8.1.a

Source-Navigator library files

/parsers/examples/assembly

Makefile
README
a.c
abrowser.l.in
b.c
build-macros
linux-i486-elf.m4
solaris-sparc.m4
toolbox.m4

 

/parsers/examples/elf

Makefile
README
blobsql.elf
ebrowser.l

Makefile for ebrowser.l
The latest release information
Example source
Parser (lex)

Parsers may be implemented in any programming language. Naturally, the task of writing a parser is significantly simpler when using compiler generation tools such as GNU flex. All of the examples provided with Source-Navigator are written using GNU flex.

A sample parser for the Elf language (an embedded SQL-like language) is provided with the SDK. To experiment with this parser, change to the .../share/sdk/parsers directory and type

make test 

This will compile the parser and then parse a number of Elf source files, placing items of interest into a project database.

The Parser Toolbox Library

A library, implemented above the API, simplifies the task of writing new parsers. This library, known as the parser toolbox, allows programmers to focus on the issues of parsing source files in their chosen language and then storing the relevant information in the database.

The toolbox provides a number of C functions that can help you write your parser with less effort. These functions can be grouped as follows:

A detailed description of the functions available in the parser toolbox library can be found in .../share/sdk/include/snptools.h.

Unless the parser has to do something out of the ordinary, it should be possible to create a new parser by following these steps:

  1. #include snptools.h in your program (or within the verbatim section of your lex specification).

  2. Define a main function which calls sn_main with the appropriate arguments and returns the result of sn_main to the host environment.
  3. Utilize the parser toolbox routines to simplify your work within the parser. For example, sn_message may be called to display messages in a dialog box that is shown to the user during the parsing process.

  4. When your parser recognizes important language constructs such as comments, function declarations, or function invocations, call the appropriate sn_insert function to insert this information into the project database.

  5. Link your final program with the sntools library. A typical command line for linking a parser can be found in the Makefile given in the .../share/sdk/parsers/examples/elf directory.

Project database calls

The toolbox library provides a number of functions for inserting information into the database that the parser will encounter in the source text. Each of these functions return an int with two possible return values:

0

success

-1

failure

sn_insert_symbol

int sn_insert_symbol(int id_type, char *classname, 
  char *identifier, char *filename, int start_lineno, 
  int startCol, int endLine, int endCol, unsigned long attrib,
  char *returnType, char *argTypes, char *argNames, 
  char *comment, int highStartLine, int highStartCol, 
  int highEndLine, int highEndCol);

sn_insert_symbol inserts a symbol into the project database.

type determines the type of the symbol. Possible values are:

SN_TYPE_DEF

type definitions (e.g. a C typedef )

SN_CLASS_DEF

class definition (particularly for object-oriented languages)

SN_MBR_FUNC_DEF

member functions

SN_MBR_VAR_DEF

member variables

SN_ENUM_DEF

enumerations

SN_CONS_DEF

constants

SN_MACRO_DEF

macros

SN_FUNC_DEF

functions

SN_SUBR_DEF

subroutines

SN_GLOB_VAR_DEF

global variables

SN_COMMON_DEF

common blocks

SN_COMMON_MBR_VAR_DEF

member variables within a common block

SN_CLASS_INHERIT

class inheritance

SN_MBR_FUNC_DCL

member function declarations

SN_FUNC_DCL

functions

SN_ENUM_CONST_DEF

enumeration constant

SN_UNION_DEF

definitions of unions or variant records

SN_FRIEND_DCL

friends (for C++)

SN_NAMESPACE_DEF

name spaces

SN_EXCEPTION_DEF

exceptions

SN_LOCAL_VAR_DEF

local variables

SN_VAR_DCL

variables

SN_INCLUDE_DEF

include files

SN_REF_UNDEFINED

reference to undefined symbols

classname

the name of the class, structure or common block of the symbol if the symbol's type is one of SN_MBR_FUNC_DEF, SN_MBR_VAR_DEF, SN_COMMON_MBR_VAR_DEF, or SN_CLASS_INHERIT. Otherwise, classname must be a NULL pointer. If the symbol's type is SN_CLASS_INHERIT, classname contains the name of the base class.

identifier

the name of the symbol to be inserted into the project database.

filename

the name of the source file in which this symbol was encountered.

start_lineno

the line number of the position where the symbol starts.

startCol

the column number of the position where the symbol starts.

endLine

the line number of the position where the symbol ends.

endCol

the column number of the position where the symbol ends.

attrib

contains attributes of the symbol definitions (see sn.h).

returnType

a string describing the return type of the function, subroutine or method. If the symbol is not one of these types, pass a NULL pointer.

argTypes

a string containing a comma-separated list of argument types for the argument list of functions, subroutines or methods. If the symbol is not one of these types, pass a NULL pointer.

argNames

a string containing a comma-separated list of argument names for the argument list of functions, subroutines or methods. If the symbol is not one of these types, pass a NULL pointer.

comment

a string containing the comment that often occurs after a definition in the source text. Note that 8-bit characters including '\n' may be used in the string. If there is no comment, pass a NULL pointer.

highStartLine

the line number of the position where the highlighting of the symbol starts.

highStartCol

the column number of the position where the highlighting of the symbol starts.

highEndLine

the line number of the position where the highlighting of the symbol ends.

highEndCol

the column number of the position where the highlighting of the symbol ends.

Examples

The following example inserts a class definition:

sn_insert_symbol(SN_CLASS_DEF, NULL, classname,
  sn_current_file(), sn_line(), sn_column(), sn_line(),
  sn_column() + strlen(classname), 0L, NULL, NULL,
  NULL, NULL, sn_line(), sn_column(), sn_line(),
  sn_column() + strlen(classname));

The following example inserts a method definition:

sn_insert_symbol(SN_MBR_FUNC_DEF, classname, methodname,
  sn_current_file(), sn_line(), sn_column(), sn_line(),
  sn_column() + strlen(methodname), 0L, NULL, NULL, 
  NULL, NULL, sn_line(), sn_column(), sn_line(), 
  sn_column() + strlen(methodname));

For a C function with the following prototype:

int transform(struct coord * p, int x, int y, unsigned att);

the following example inserts the definition into the database:

sn_insert_symbol(SN_FUNC_DEF, NULL, "transform",
  sn_current_file(), sn_line(), sn_column(), sn_line(),
  sn_column + len, 0L, "int", "struct coord *, int, int,
  unsigned", "p,x,y,att", NULL, NULL, sn_line(), sn_column(),
  sn_line(), sn_column() + len);

sn_insert_xref

sn_insert_xref inserts cross-referencing information.

One of the most useful aspects of Source-Navigator is its cross-referencing capabilities. For instance, it is possible to see which functions are used by other functions or which functions modify a particular global variable.

Where possible, a parser should attempt to collect information of this nature and insert it into the project database. In a language with a flat namespace such as C, this can be achieved by noting the name of the current function within the parser. If a function invocation is encountered in the source text, then the cross-referencing can be inferred based on the current function's name and the function being called.

Cross-referencing information is added using:

int sn_insert_xref(int type, int scope_type, int scope_level,
  char *classname, char *funcname, char *argtypes, 
  char *refclass, char *refsymbol, char *ref_arg_types, 
  char *filename, int lineno, int acc);

type

describes the type of the referenced symbol. It must be one of the following:

 

SN_REF_TO_TYPEDEF

 

SN_REF_TO_DEFINE

 

SN_REF_TO_ENUM

 

SN_REF_TO_STRUCT

 

SN_REF_TO_UNION

 

SN_REF_TO_CLASS

 

SN_REF_TO_FUNCTION

 

SN_REF_TO_MBR_FUNC

 

SN_REF_TO_MBR_VAR

 

SN_REF_TO_COMM_VAR

 

SN_REF_TO_CONSTANT

 

SN_REF_TO_SUBROUTINE

 

SN_REF_TO_GLOB_VAR

 

SN_REF_TO_LOCAL_VAR

 

SN_REF_TO_TEMPLATE

 

SN_REF_TO_NAMESPACE

 

SN_REF_TO_EXCEPTION

 

SN_REF_TO_LABEL

scope_type

describes the type of the location where the cross-reference information is reported. It must be one of the following:

SN_FUNC_DEF
SN_MBR_FUNC_DEF
SN_SUBR_DEF

scope_level

describes the scope level of the referenced symbol. It must be one of:

SN_REF_SCOPE_LOCAL
SN_REF_SCOPE_GLOBAL

classname

a string containing the class name of the method if scope_type is SN_MBR_FUNC_DEF; otherwise, it must be a NULL pointer.

funcname

a string containing the function, method or subroutine name in which the reference information is reported.

argtypes

a string containing a comma-separated list of argument types for the argument list of functions, subroutines or methods. Pass NULL if there are no arguments.

refclass

a string containing the class, structure or common block name of the referred symbol. If the symbol is not within a namespace, pass a NULL pointer.

refsymbol

the name of the referred symbol.

ref_arg_types

a string containing a comma-separated list of argument types if the referred symbol's type is a method, subroutine or function. Pass NULL if there are no arguments or the referred symbol is not one of these types.

filename

the name of the source file in which the reference information is reported.

lineno

the line number of the source file in which the reference information is reported.

acc

the level of access to the referenced symbol and is one of:

 

SN_REF_READ

symbol is read (e.g. if (x) { ... })

 

SN_REF_WRITE

symbol is modified (e.g. x = 10 )

 

SN_REF_PASS

variable is passed to a subroutine/function

 

SN_REF_UNUSED

symbol is never used

Examples

The following example inserts cross-referencing information for a function that is called from another function.

sn_insert_xref(SN_REF_TO_FUNCTION, SN_FUNC_DEF,
  SN_REF_SCOPE_GLOBAL, NULL, currentFunction, NULL, NULL,
  calledFunction, NULL, sn_current_file(), sn_line(),
  SN_REF_PASS);

The following example inserts cross-referencing information for a function that is called from a member function called insert which belongs to a C++ class called Stack.

sn_insert_xref(SN_REF_TO_MBR_VAR, SN_MBR_FUNC_DEF,
  SN_REF_SCOPE_GLOBAL, "Stack", "insert", NULL, "Stack", 
  "i", NULL, sn_current_file(), sn_line(), SN_REF_READ);

sn_insert_comment

sn_insert_comment inserts comments into the project database.

When comments are encountered in the source text, the parser should call sn_insert_comment to add these comments to the project database. In some Source-Navigator projects, the user will choose to not include comments, but the parser should call this function regardless. The library function will decide whether or not to actually store the information in the database.

Comments are added to the database using:

int sn_insert_comment(char *classname, char *funcname, 
  char *filename, char *comment, int beg_line, int beg_col);

classname

a string containing the name of the class or method where the comment was found or NULL.

funcname

a string containing the name of the function or method in which the comment was found, or NULL if the comment is outside any function or method scope.

filename

a string containing the name of the current file being parsed.

comment

a string containing the comment without the comment separators. For example, in Tcl this would exclude the leading "#" character.

beg_line

the line number of the source file where the comment begins.

beg_col

the column number of the source file where the comment begins.

Integration with Source-Navigator

A completed parser can be integrated into Source-Navigator by following these steps:

  1. Copy the parser executable into the .../bin directory.

  2. Edit sn_prop.cfg in the directory .../share/etc. The Tcl procedure sn_add_parser is used to add parsers to the configuration. The following example shows how to include support for Java:

    sn_add_parser java -suffix {*.java} \
      -brow_cmd $odd_path(bindir)/jbrowser \
      -high_cmd $odd_path(bindir)/jbrowser \
      -high_switch "-h"

When Source-Navigator is restarted, support for the new language becomes available. When creating a project, the Parsers tab in the Project Preferences dialog shows the new language and its associated filename extensions.

This is all that is required to add new language support to Source-Navigator. If a project is created which contains files with any of the specified filename extensions, the new parser is invoked to process those files as a part of the overall parsing process.


Contents

Previous

Next