optscript

For seven years, @masatake has made the execution of ctags slower. He is working hard for implementing yet another feature called optscript that makes the execution much slower.


optscript is an implementation of PostScript alike stack-oriented programming language. optscript has many non-graphical operators already.

You can use optscript to add extra actions to --regex-<LANG>= option by putting optscript code at the end of the option. You must wrap the code with {{ and }}.

Let’s see an example.

input source code (input.hello):

def x-y-z:
end

optlib file (example.ctags):

--langdef=hello
--kinddef-hello=d,def,definitions
--map-hello=+.hello

--regex-hello=/^def +([-a-z]+):$/\1/d/{{
	/putlast { 1 index length exch put } def
	/tr {
	    % str [int<from> int<to>] str'
	    ()
	    2 index {
		% str [int<from> int<to>] str' int<chr>
		dup 3 index
		% str [int<from> int<to>] str' int<chr>  int<chr> [int<from> int<to>]
		0 get
		% str [int<from> int<to>] str' int<chr> int<chr> int<from>
		eq {
		    pop
		    % str [int<from> int<to>] str'
		    dup 2 index 1 get putlast
		} {
		    % str [int<from> int<to>] str' int<chr>
		    1 index exch putlast
		} ifelse
	    } forall
	    % str [int<from> int<to>] str'
	    exch pop
	    0 exch putinterval
	} def
	. :name {
	   dup (-_) tr
	   . exch name:
	} if
}}

ctags execution:

$ u-ctags --options=example.ctags -o - input.hello
x_y_z	input.hello	/^def x-y-z:$/;"	d

In the example Universal-ctags with example.ctags extracts x-y-z in input.hello.

example.ctags transforms the tag name x-y-z to x_y_z. The code written in optscript does this transformation.

Let’s look the code between --regex-hello=/^def +([-a-z]+):$/\1/d/{{ and }}.

	/putlast { 1 index length exch put } def

This fragment defines a procedure named putlast. putlast put a character at the end of a string buffer.

	/tr {
     ...
	} def

This defines a procedure named tr. tr replaces translates characters.

(a_b_c) (_-) tr

In the above example, tr replaces _ in a_b_c with -. As a result, you get a-b-c.

Defining putlast and tr is just for the preparation for the next step.

	. :name {
	   dup (-_) tr
	   . exch name:
	} if

. represents a tag entry (tagEntryInfo) for x-y-z extracted by the regular expression pattern /^def +([-a-z]+):$/. The tag entry having x-y-z as name, and d as kind is already built on the memory but ctags doesn’t emit it to a tags file yet because ctags must execute the action specified with {{...}} before emitting.

. :name gets the name of the tag entry as a string and put it to the operand stack of optscript interpreter. (-_) tr replaces - in the string with _. . exch name: sets the string to the name field of the tag entry.

After executing the optscript code, ctags emits the tag entry to the tag file. What you will see is the transformed tag entry:

x_y_z	input.hello	/^def x-y-z:$/;"	d

I will add more operators to export the ctags internal APIs that could be used only from parsers (crafted parsers) written in C language. Providing operators to access the scope stack is the primary target. It will be an important part of Vue parser. See https://github.com/universal-ctags/ctags/issues/1577 about the background.

Do you want to use optscript in your .ctags? You can learn the language with BlueBook till I (@masatake) merge the change for the optscript interpreter to ctags.

K&R is a book for persons who think that studying the stack-oriented languge is too hard.

Snaps now built for more architectures

Snaps now built for more architectures

Linux installs using Snaps added support for more architectures, including:

arm64, armhf, i386, ppc64el, and s390x.

The new architectures are released to the ‘edge’ channel for now. You can install them on devices of those architectures, such as Pis, using:

snap install --edge universal-ctags

File an issue if there are any problems, or to tell us that they work fine!

If we see download stats for these versions with no issues reported, then we’ll promote them to the stable channel in a few days.

The existing amd64 architecture also received the same update, which has been tested and promoted to the stable channel already. Updates should roll out imminently to those of our 2,000 existing Snap users who have auto-updates enabled.

The status of the development

Weekly git tagging

@masatake is implementing weekly automatic git tagging run at GitHub Actions. Many people, especially binary packagers, have wanted official version tags. Tag names will follow the form: v5.9.YYYYMMDD.0.

Updated: The form of the tag names has been changed to p5.9.YYYYMMDD.0, where “p” means periodical.

The action will not create a tag if no change is committed within a week after the last tagging.

New maintainer for (System)Verilog parser

@hirooih joins the team as the maintainer of (System)Verilog parser. @hirooih solved a long standing issue of the parser: supporting defined types.

Julia parser

@getzze opened an pull request for adding a parser for Julia Programming Language. @getzze and @masatake are working on merging it to u-ctags.

Snaps are available.

Universal-ctags has long been available as a snap. For a short while, the snap wasn’t actively being maintained, but now it has an active maintainer again.

Recent snap fixes include adding an automatically applied alias of ‘ctags’ when the snap is installed, adding an end-to-end test to check the snap builds and installs a working application, and adding CI using Travis.

Applications installed as a snap are subject to security restrictions, which can introduce limitations compared to installing as an apt or compiled from source.

Our snap’s limitations are now documented on universal-ctags’ page in the snapstore, and the new snap maintainer is working to reduce or eliminate them.

Currently the biggest limitation is that a project’s .ctags.d/ config directory cannot be read from the current directory. This can be worked around by moving it to a non-hidden directory name, and using --options to explicitly tell ctags to use it. The user’s ~/.ctags.d/ directory is read as expected.

Otherwise, the snap is viable for daily use, and is currently installed by over 500 users and growing.

If you have any thoughts on those limitations, or on other issues you find with the snap, let us know.

New Website

We’ve got a new website! We’ll be updating the blog with development news over the next few months, so stay tuned!

In the meantime, if you’re doing interesting things with Universal Ctags and you’d like to write about it, let us know and we’ll work with you on getting it published on the blog!