README.md

hl_tcl

The hl_tcl package is a syntax highlighter for Tcl/Tk code.

It can be applied to a Tk text widget or to a static html page.

The Tk text widget may be made read-only or editable. Also, the hl_tcl may take an argument, sort of command to watch the viewing / editing.

When applied to html pages, the hl_tcl highlights Tcl/Tk code snippets embedded between <code> </code> tags.

The hl_tcl has highlighted its own code in Reference.

Some of blah-blah

The Tcl being incredibly dynamic language sets a lot of problems before any Tcl syntax highlighter. Probably, the usage of quotes and esp. the strings spanning several lines are the main challenges.

Below is a line that brings most (not hl_tcl, as seen in Reference) of Tcl highlighters in a stupor:

if {[set i [string first {"} $line $i]]==-1} {return no}

... as well as this one:

regsub -all {(([^A-Z@]|\\@)[.?!]("|'|'')?([])])?) } $fieldText {\1  } fieldText

Good luck for a highlighter when the second line (or similar) follows the first, giving it a matching quote and thus bringing it out of the stupor.

Those orphan quotes are often used in regexp and regsub Tcl commands, so that when a honest Tcl highlighter (like Geany) stumbles upon an orphan quote, it tries its best to highlight the rest of code as a string, till the next unmatched quote.

Thus, we have

... instead of

There are "tricky" highlighters (like Gedit) that behave more wisely at the stumbling an orphan quote: they permit only a one-line Tcl strings (if not continued with \\), so that the string highlighting would be most likely finished in the same line it started. No problems except for this silly line. And no delays due to the highlighting the rest of code...

... as seen in:

Some of editors

Geany. Probably, the best Tcl highlighter. And the great programming tool at that. Still, it has few drawbacks:

doesn't highlight the above mentioned Tcl lines properly

doesn't highlight ${var} in contrast with $var

tries to highlight any (even hexidecimal) number it encounters, thus set a 1fix or set b #abxxx looks a bit peculiar

set c {{#000} #FFF} is quite a legal Tcl command as well as set c {#000 #FFF}, not for Geany

no highlighting TclOO (method, my, mixin etc.)

Vim. Probably, the fastest Tcl highlighter. Great and awful. Nonetheless:

tricky with those above mentioned Tcl lines

doesn't highlight ttk commands (Tk only) and TclOO

tries to highlight every bit of Tcl, e.g. set set set is highlighted as three set commands ;)

as a result, much more florid than most of others

Kate. As nearly good as Geany. As nearly florid as Vim (set set set). Doesn't highlight ttk and TclOO.

TKE. Written in Tcl/Tk, it might be the best of all to highlight the Tcl/Tk. In spite of its suspended state it still can. Issues with highlighting strings and the performance.

Pluma and Gedit seem to use the same Tcl highlighting engine that gives rather good results. Still, the mentioned above drawbacks are here too. And no highlighting of tk, ttk, TclOO.

Notepad++. Very fast Tcl highlighter. And very basic. All the same drawbacks. No highlighting of tk, ttk, TclOO. Plus an obsolete version of Tcl, i.e. no highlighting lset, lassign etc.

What can we do?

To develop an ideal (correct and fast) Tcl/Tk highlighter, we would have to dive into Tcl core. Though, no hopes to achieve the ideal through repeating the core in Tcl/Tk or massively using the regular expressions.

That said, while implementing Tcl/Tk highlighter in pure Tcl/Tk, we might hope to achieve a reasonable compromise between the performance and the elimination of blunders.

It seems hl_tcl got close to this compromise. Specifically, it provides:

special highlighting for Tcl and TclOO commands

special highlighting for Tk and ttk commands

allowing additional commands to highlight (as Tk ones)

special highlighting for declarations proc, method, oo::class etc. as well as return, yield

special highlighting for #comments, $variables, "strings", -options

in-line comments being recognized and thus highlighted only after ;#

proper handling of most regexp and regsub expressions containing a quote

highlighting the multi-line strings, with possible switching this mode off (a-la Gedit) to improve the performance

customizing colors of the highlighting

highlighting viewable/editable Tk text widget and static html pages

good performance at editing 1000-4000 LOC and rather acceptable for 4000-9000 LOC

even monstrous 10000 LOC and more are handled fast at the "tricky" mode a-la Gedit

The hl_tcl doesn't provide the following:

highlighting numbers

highlighting brackets, except for matched ones and inside the strings

These are in no way critical drawbacks. A little less florid Tcl code might be even preferable for other tastes.

The Tcl can arrange its pitfalls for hl_tcl (I know where). Also, tricky practices or tastes can make a fool of hl_tcl. Still hopefully these pranks are few and rare to encounter.

Use for text widget

The code below:

package require hl_tcl

proc ::stub {args} {puts "stub: $args"}

::hl_tcl::hl_init $::txt -readonly yes -cmd ::stub

#... inserting a text into the text widget

::hl_tcl::hl_text $::txt

sets an example of hl_tcl usage. Here are the details:

::stub is a procedure to watch the text editing; here it simply puts out the text's last index;

hl_init is called before filling the text widget with a Tcl code; it sets the highlighting options and disables the highlighting till hl_text runs;

hl_text runs to highlight the Tcl code of the text widget and to view/edit it.

The hl_init takes arguments:

txt is the text widget's path

args contains options of text widget (omittable)

The args is a list of -option "value" where -option may be:

-colors - list of colors: clrCOM, clrCOMTK, clrSTR, clrVAR, clrCMN, clrPROC, clrOPT

-dark - flag "dark background of text", i.e. simplified -colors (default "no")

-font - attributes of text font

-readonly - flag "text is read-only" (default "no")

-multiline - flag "multi-line strings" (default "yes")

-cmd - command to watch editing/viewing (default "")

-cmdpos - command to watch cursor positioning (default "")

-seen - number of first lines seen at start (default 500)

-optRE - flag "use a regular expression to highlight options" (default "yes")

-keywords - additional commands to highlight (as Tk ones)

-dobind - if true, forces keys binding at repeating calls of hl_init

-plaintext - true for plain texts with no highlighting

-plaincom - a command for plain highlighting line by line

Note: -seen 500 and -multiline no can improve the performance a lot. It's recommended to use -seen 500 (or any other reasonable limit, e.g. -seen 200) at any rate, except for static html pages.

A command for -plaincom option has two arguments: a current text's path and a current line's number. It should highlight the current line and return true, otherwise (if the current line is Tcl code) it returns false. An example of its usage is presented by alited editor (lib/addon directory).

The rest of hl_tcl procedures are:

hl_all updates all highlighted existing text widgets, e.g. at changing a color scheme of application

hl_readonly gets/sets a read-only mode and/or a command to watch a text widget at viewing/editing it

hl_colors gets a list of colors for highlighting

See details in Reference.

Use for static html

In the hl_tcl.zip, there is a Tcl script named tcl_html.tcl that highlights Tcl snippets of static html page(s).

It runs as follows:

tclsh tcl_html.tcl "glob-pattern-of-html-files"

For example:

tclsh ~/UTILS/hl_tcl/tcl_html.tcl "~/UTILS/mulster/tasks/ruff/src/*"

In this example, the html files are located in ~/UTILS/mulster/tasks/ruff/src.

Perhaps, you would want to modify the tcl_html.tcl, this way:

replace "no" with "yes" for dark html pages

replace <code class="tcl"> with html tags starting the Tcl code in your html files

replace </code> with html tags finishing the Tcl code in your html files

These are arguments of ::hl_tcl_html::highlight procedure.

The tag pairs can be multiple if the html pages contain them, e.g.

::hl_tcl_html::highlight $fhtml "no" \
  {<code class="tcl">} {</code>} \
  {<pre class="code">} {</pre>}

Links

Reference

Source (hl_tcl.zip)

Demo of hl_tcl v0.6.1 (25 Mb)

Note that hl_tcl is still disposed to update.

hl_tcl_html.tcl

# _______________________________________________________________________ #
#
# Highlighting Tcl code with html tags.
#
# Scripted by Alex Plotnikov (aplsimple@gmail.com).
# License: MIT.
# _______________________________________________________________________ #

package require Tk
source [file join [file dirname [info script]] hl_tcl.tcl]

namespace eval ::hl_tcl_html {
}

proc  ::hl_tcl_html::insertTag {pN tN lcodeN} {
  # Inserts a html tag into Tcl code.
  #   pN - variable's name for a position of the tag
  #   tN - variable's name for the tag
  #   lcodeN - variable's name for the list of code lines

  upvar 1 $pN p $tN t $lcodeN lcode
  lassign [split $p .] l c
  incr l -1
  set line [lindex $lcode $l]
  set line1 [string range $line 0 $c-1]
  set line2 [string range $line $c end]
  set lcode [lreplace $lcode $l $l "$line1$t$line2"]
}

proc ::hl_tcl_html::highlight {htmlfile darkedit args} {
  # Processes html file to find and highlight embedded Tcl code.
  #   htmlfile - file name
  #   darkedit - flag "the text widget has dark background" ("no" by default)
  #   args - list of tag pairs
  # A tag pair consists of:
  #   tag1 - opening tag(s) of Tcl code snippet
  #   tag2 - ending tag(s) of Tcl code snippet

  set txt .t
  text $txt
  set chan [open $htmlfile]
  chan configure $chan -encoding utf-8
  set text [read $chan]
  close $chan
  lassign [::hl_tcl::hl_colors 1 $darkedit] clrCOM clrCOMTK clrSTR clrVAR clrCMN clrPROC clrOPT
  lassign [::hl_tcl::addingColors $darkedit] -> clrCMN2
  set cs {}
  foreach {tag1 tag2} $args {
    if {[string match -nocase cs $tag1]} {
      lassign [split $tag2 ,] clrCOM clrCOMTK clrSTR clrVAR clrCMN clrPROC clrOPT
      continue
    }
    set ic [set ic2 0]
    while {$ic>=0 && $ic2>=0} {
      set ic [string first $tag1 $text $ic]
      if {$ic>=0} {
        incr ic [string length $tag1]
        set ic2 [string first $tag2 $text $ic]
        if {$ic2>=0} {
          set code [string range $text $ic $ic2-1]
          if {[regexp "<font color=#\[0-9a-fA-F\]{6}>" $code]} {
            set ic [expr {$ic2+[string length $tag2]}]
            continue  ;# already processed
          }
          set code [string map [list """ \" "&" & < "<" > ">"] $code]
          ::hl_tcl::hl_init $txt -dark $darkedit -seen 99999999
          $txt replace 1.0 end $code
          ::hl_tcl::hl_text $txt
          set taglist [list]
          foreach tag {tagCOM tagCOMTK tagSTR tagVAR tagCMN tagCMN2 tagPROC tagOPT} {
            foreach {p1 p2} [$txt tag ranges $tag] {
              lassign [split $p1 .] l1 c1
              lassign [split $p2 .] l2 c2
              lappend taglist [list [format %06d $l1][format %06d $c1] $tag 1 $p1]
              lappend taglist [list [format %06d $l2][format %06d $c2] $tag 2 $p2]
            }
          }
          set taglist [lsort -decreasing $taglist]
          set lcode [split $code \n]
          foreach tagdat $taglist {
            lassign $tagdat -> tag typ pos
            switch -exact $tag {
              tagCOM {
                set t1 "<b><font color=$clrCOM>"
                set t2 "</font></b>"
              }
              tagCOMTK {
                set t1 "<b><font color=$clrCOMTK>"
                set t2 "</font></b>"
              }
              tagPROC {
                set t1 "<b><font color=$clrPROC>"
                set t2 "</font></b>"
              }
              tagSTR {
                set t1 "<font color=$clrSTR>"
                set t2 "</font>"
              }
              tagVAR {
                set t1 "<font color=$clrVAR>"
                set t2 "</font>"
              }
              tagCMN {
                set t1 "<i><font color=$clrCMN>"
                set t2 "</font></i>"
              }
              tagCMN2 {
                set t1 "<i><font color=$clrCMN2>"
                set t2 "</font></i>"
              }
              tagOPT {
                set t1 "<font color=$clrOPT>"
                set t2 "</font>"
              }
            }
            if {$typ==1} {
              insertTag pos t1 lcode
            } else {
              insertTag pos t2 lcode
            }
          }
          set code ""
          foreach lc $lcode {
            if {$code ne ""} {append code \n}
            append code $lc
          }
          set code [string map [list \" """] $code]
          set text1 [string range $text 0 $ic-1]
          set text2 [string range $text $ic2 end]
          set text "$text1$code"
          set ic [string length $text]
          set text "$text$text2"
        }
      }
    }
  }
  set chan [open $htmlfile w]
  chan configure $chan -encoding utf-8
  puts -nonewline $chan $text
  close $chan
}
after idle exit
# _________________________________ EOF _________________________________ #
#% file copy -force .bak/index-SRC.html .bak/index.html
#% exec tclsh ./tcl_html.tcl .bak/index.html
#% exec opera .bak/index.html

hl_tcl_html.tcl