Shallow Thoughts : tags : editors

Insert HTML Tags in Emacs

Emacs has various options for editing HTML, none of them especially good. I gave up on html-mode a while back because it had so many evil tendencies, like not ever letting you type a double dash without reformatting several lines around the current line as an HTML comment. I'm using web-mode now, which is better.

But there was one nice thing that html-mode had: quick key bindings for inserting tags. For instance, C-c i would insert the tag for italics, <i></i>, and if you had something selected it would make that italic: <i>whatever you had selected</i>.

It's a nice idea, but it was too smart (for some value of "smart" that means "dumb and annoying") for its own good. For instance, it loves to randomly insert things like newlines in places where they don't make any sense.

Read more ...

Tags: editors, emacs, html, html-mode
[ 19:07 Nov 26, 2021    More linux/editors | permalink to this entry | ]

Key Bindings for Copy and Paste Under X11

In the previous article I wrote about how the two X selections, the primary and clipboard, work. But I glossed over the details of key bindings to copy and paste the two selections in various apps.

That's because it's complicated. Although having the two selections available is really a wonderful feature, I can understand why so many people are confused about it or think that copy and paste just generally doesn't work on Linux -- because apps are so woefully inconsistent about their key bindings, and what they do have is so poorly documented.

"Why don't they all just use the standard Ctrl-C, Ctrl-V?" you ask. (Cmd-C, Cmd-V for Mac users, but I'm not going to try to include the Mac versions of every binding in this article; Mac users will have to generalize.)

Simple: those keys have well-known and very commonly used bindings already, which date back to long before copy and paste were invented.

Read more ...

Tags: linux, X11, cmdline, emacs, vim, editors
[ 12:54 Sep 11, 2020    More linux | permalink to this entry | ]

Emacs: Typing dashes in html mode (update for Emacs 24)

Back in 2006, I wrote an article on making a modified copy of sgml-mode.el to make it possible to use double-dashed clauses -- like this -- in HTML without messing up auto-fill mode.

That worked, but the problem is that if you use your own copy of sgml-mode.el, you miss out on any other improvements to HTML and SGML mode. There have been some good ones, like smarter rewrap of paragraphs. I had previously tried lots of ways of customizing sgml-mode without actually replacing it, but never found a way.

Now, in emacs 24.5.1, I've found a easier way that seems to work. The annoying mis-indentation comes from the function sgml-comment-indent-new-line, which sets variables comment-start, comment-start-skip and comment-end and then calls comment-indent-new-line.

All I had to do was redefine sgml-comment-indent-new-line to call comment-indent-new-line without first defining the comment characters:

(defun sgml-comment-indent-new-line (&optional soft)
  (comment-indent-new-line soft))

Finding emacs source

I wondered if it might be better to call whatever underlying indent-new-line function comment-indent-new-line calls, or maybe just to call (newline-and-indent). But how to find the code of comment-indent-new-line?

Happily, describe-function (on C-h f, or if like me you use C-h for backspace, try F-1 h) tells you exactly what file defines a function, and it even gives you a link to click on to view the source. Wonderful!

It turned out just calling (newline-and-indent) wasn't enough, because sgml-comment-indent-new-line typically calls comment-indent-new-line when you've typed a space on the end of a line, and that space gets wrapped and then messes up indentation. But you can fix that by copying just a couple of lines from the source of comment-indent-new-line:

(defun sgml-comment-indent-new-line (&optional soft)
  (save-excursion (forward-char -1) (delete-horizontal-space))
  (delete-horizontal-space)
  (newline-and-indent))

That's a little longer than the other definition, but it's cleaner since comment-indent-new-line is doing all sorts of extra work you don't need if you're not handling comments. I'm not sure that both of the delete-horizontal-space lines are needed: the documentation for delete-horizontal-space says it deletes both forward and backward. But I have to assume they had a good reason for having both: maybe the (forward-char -1) is to guard against spurious spaces already having been inserted in the next line. I'm keeping it, to be safe.

Tags: emacs, editors
[ 11:16 Jun 09, 2017    More linux/editors | permalink to this entry | ]

Highlight and remove extraneous whitespace in emacs

I recently got annoyed with all the trailing whitespace I saw in files edited by Windows and Mac users, and in code snippets pasted from sites like StackOverflow. I already had my emacs set up to indent with only spaces:

(setq-default indent-tabs-mode nil)
(setq tabify nil)

and I knew about M-x delete-trailing-whitespace ... but after seeing someone else who had an editor set up to show trailing spaces, and tabs that ought to be spaces, I wanted that too.

To show trailing spaces is easy, but it took me some digging to find a way to control the color emacs used:

;; Highlight trailing whitespace.
(setq-default show-trailing-whitespace t)
(set-face-background 'trailing-whitespace "yellow")

I also wanted to show tabs, since code indented with a mixture of tabs and spaces, especially if it's Python, can cause problems. That was a little harder, but I eventually found it on the EmacsWiki: Show whitespace:

;; Also show tabs.
(defface extra-whitespace-face
  '((t (:background "pale green")))
  "Color for tabs and such.")

(defvar bad-whitespace
  '(("\t" . 'extra-whitespace-face)))

While I was figuring this out, I got some useful advice related to emacs faces on the #emacs IRC channel: if you want to know why something is displayed in a particular color, put the cursor on it and type C-u C-x = (the command what-cursor-position with a prefix argument), which displays lots of information about whatever's under the cursor, including its current face.

Once I had my colors set up, I found that a surprising number of files I'd edited with vim had trailing whitespace. I would have expected vim to be better behaved than that! But it turns out that to eliminate trailing whitespace, you have to program it yourself. For instance, here are some recipes to Remove unwanted spaces automatically with vim.

Tags: editors, emacs
[ 16:41 Feb 18, 2017    More linux/editors | permalink to this entry | ]

Emacs: Initializing code files with a template

Part of being a programmer is having an urge to automate repetitive tasks.

Every new HTML file I create should include some boilerplate HTML, like <html><head></head></body></body></html>. Every new Python file I create should start with #!/usr/bin/env python, and most of them should end with an if __name__ == "__main__": clause. I get tired of typing all that, especially the dunderscores and slash-greater-thans.

Long ago, I wrote an emacs function called newhtml to insert the boilerplate code:

(defun newhtml ()
  "Insert a template for an empty HTML page"
  (interactive)
  (insert "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\">\n"
          "<html>\n"
          "<head>\n"
          "<title></title>\n"
          "</head>\n\n"
          "<body>\n\n"
          "<h1></h1>\n\n"
          "<p>\n\n"
          "</body>\n"
          "</html>\n")
  (forward-line -11)
  (forward-char 7)
  )

The motion commands at the end move the cursor back to point in between the <title> and </title>, so I'm ready to type the page title. (I should probably have it prompt me, so it can insert the same string in title and h1, which is almost always what I want.)

That has worked for quite a while. But when I decided it was time to write the same function for python:

(defun newpython ()
  "Insert a template for an empty Python script"
  (interactive)
  (insert "#!/usr/bin/env python\n"
          "\n"
          "\n"
          "\n"
          "if __name__ == '__main__':\n"
          "\n"
          )
  (forward-line -4)
  )

... I realized that I wanted to be even more lazy than that. Emacs knows what sort of file it's editing -- it switches to html-mode or python-mode as appropriate. Why not have it insert the template automatically?

My first thought was to have emacs run the function upon loading a file. There's a function with-eval-after-load which supposedly can act based on file suffix, so something like (with-eval-after-load ".py" (newpython)) is documented to work. But I found that it was never called, and couldn't find an example that actually worked.

But then I realized that I have mode hooks for all the programming modes anyway, to set up things like indentation preferences. Inserting some text at the end of the mode hook seems perfectly simple:

(add-hook 'python-mode-hook
          (lambda ()
            (electric-indent-local-mode -1)
            (font-lock-add-keywords nil bad-whitespace)
            (if (= (buffer-size) 0)
                (newpython))
            (message "python hook")
            ))

The (= (buffer-size) 0) test ensures this only happens if I open a new file. Obviously I don't want to be auto-inserting code inside existing programs!

HTML mode was a little more complicated. I edit some files, like blog posts, that use HTML formatting, and hence need html-mode, but they aren't standalone HTML files that need the usual HTML template inserted. For blog posts, I use a different file extension, so I can use the elisp string-suffix-p to test for that:

  ;; s-suffix? is like Python endswith
  (if (and (= (buffer-size) 0)
           (string-suffix-p ".html" (buffer-file-name)))
      (newhtml) )

I may eventually find other files that don't need the template; if I need to, it's easy to add other tests, like the directory where the new file will live.

A nice timesaver: open a new file and have a template automatically inserted.

Tags: emacs, editors, programming
[ 09:52 Feb 13, 2017    More linux/editors | permalink to this entry | ]

Visual diffs and file merges with vimdiff

I needed to merge some changes from a development file into the file on the real website, and discovered that the program I most often use for that, meld, is in one of its all too frequent periods where its developers break it in ways that make it unusable for a few months. (Some of this is related to GTK, which is a whole separate rant.)

That led me to explore some other diff/merge alternatives. I've used tkdiff quite a bit for viewing diffs, but when I tried to use it to merge one file into another I found its merge just too hard to use. Likewise for emacs: it's a wonderful editor but I never did figure out how to get ediff to show diffs reliably, let alone merge from one file to another.

But vimdiff looked a lot easier and had a lot more documentation available, and actually works pretty well.

I normally run vim in an xterm window, but for a diff/merge tool, I want a very wide window which will show the diffs side by side. So I used gvimdiff instead of regular vimdiff: gvimdiff docs.dev/filename docs.production/filename

Configuring gvimdiff to see diffs

gvimdiff initially pops up a tiny little window, and it ignores Xdefaults. Of course you can resize it, but who wants to do that every time? You can control the initial size by setting the lines and columns variables in .vimrc. About 180 columns by 60 lines worked pretty well for my fonts on my monitor, showing two 80-column files side by side. But clearly I don't want to set that in .vimrc so that it runs every time I run vim; I only want that super-wide size when I'm running a side-by-side diff.

You can control that by checking the &diff variable in .vimrc:

if &diff
    set lines=58
    set columns=180
endif

If you do decide to resize the window, you'll notice that the separator between the two files doesn't stay in the center: it gives you lots of space for the right file and hardly any for the left. Inside that same &diff clause, this somewhat arcane incantation tells vim to keep the separator centered:

    autocmd VimResized * exec "normal \<C-w>="

I also found that the colors, in the vim scheme I was using, made it impossible to see highlighted text. You can go in and edit the color scheme and make your own, of course, but an easy way quick fix is to set all highlighting to one color, like yellow, inside the if $diff section:

    highlight DiffAdd    cterm=bold gui=none guibg=Yellow
    highlight DiffDelete cterm=bold gui=none guibg=Yellow
    highlight DiffChange cterm=bold gui=none guibg=Yellow
    highlight DiffText   cterm=bold gui=none guibg=Yellow

Merging changes

Okay, once you can view the differences between the two files, how do you merge from one to the other? Most online sources are quite vague on that, but it's actually fairly easy:

]c	jumps to the next difference
[c	jumps to the previous difference
dp	makes them both look like the left side (apparently stands for diff put
do	makes them both look like the right side (apparently stands for diff obtain

The only difficult part is that it's not really undoable. u (the normal vim undo keystroke) works inconsistently after dp: the focus is generally in the left window, so u applies to that window, while dp modified the right window and the undo doesn't apply there. If you put this in your .vimrc

nmap du :wincmd w<cr>:normal u<cr>:wincmd w<cr>

then you can use du to undo changes in the right window, while u still undoes in the left window. So you still have to keep track of which direction your changes are going.

Worse, neither undo nor this du command restores the highlighting showing there's a difference between the two files. So, really, undoing should be reserved for emergencies; if you try to rely on it much you'll end up being unsure what has and hasn't changed.

In the end, vimdiff probably works best for straightforward diffs, and it's probably best get in the habit of always merging from right to left, using do. In other words, run vimdiff file-to-merge-to file-to-merge-from, and think about each change before doing it to make it less likely that you'll need to undo.

And hope that whatever silly transient bug in meld drove you to use vimdiff gets fixed quickly.

Tags: linux, editors, vim
[ 20:10 Jun 09, 2016    More linux/editors | permalink to this entry | ]

Emacs rich-text mode: coloring and styling plain text

I use emacs a lot for taking notes, during meetings, while watching lectures in a MOOC, or while researching something.

But one place where emacs falls short is highlighting. For instance, if I paste a section of something I'm researching, then I want to add a comment about it, to differentiate the pasted part from my added comments, I have to resort to horrible hacks like "*********** My comment:". It's like the stuff Outlook users put in emails because they can't figure out how to quote.

What I really want is a simple rich-text mode, where I can highlight sections of text by changing color or making it italic, bold, underlined.

Enter enriched-mode. Start it with M-x enriched-mode and then you can apply some styles with commands like M-o i for italic, M-o b for bold, etc. These styles may or may not be visible depending on the font you're using; for instance, my font is already bold and emacs isn't smart enough to make it bolder, the way some programs are. So if one style doesn't work, try another one.

Enriched mode will save these styles when you save the file, with a markup syntax like <italic>This text is in italic.</italic> When you load the file, you'll just see the styles, not the markup.

Colors

But they're all pretty subtle. I still wanted colors, and none of the documentation tells you much about how to set them.

I found a few pages saying that you can change the color of text in an emacs buffer using the Edit menu, but I hide emacs's menus since I generally have no use for them: emacs can do everything from the keyboard, one of the things I like most about it, so why waste space on a menu I never use? I do that like this:

(tool-bar-mode 0)
(menu-bar-mode 0)

It turns out that although the right mouse button just extends the selection, Control-middleclick gives a context menu. Whew! Finally a way to change colors! But it's not at all easy to use: Control-middleclick, mouse over Foreground Color, slide right to Other..., click, and the menu goes away and now there's a prompt in the minibuffer where you can type in a color name.

Colors are saved in the file with a syntax like: <x-color><param>red</param>This text is in red.</x-color>

All that clicking is a lot of steps, and requires taking my hands off the keyboard. How do I change colors in an easier, keyboard driven way? I drew a complete blank with my web searches. A somewhat irritable person on #emacs eventually hinted that I should be using overlays, and I eventually figured out how to set overlay colors ((overlay-put (make-overlay ...)) turned out to be the way to do that) but it was a complete red herring: enriched-mode doesn't pay any attention to overlay colors. I don't know what overlays are useful for, but it's not that.

But in emacs, you can find out what's bound to a key with describe-key. Maybe that works for mouse clicks too? I ran describe-key, held down Control, clicked the middle button -- the context menu came up -- then navigated to Foreground Color and Other... and discovered that it's calling (facemenu-set-foreground COLOR &optional START END).

Binding to keys

Finally, a function I can bind to a key! COLOR is just a string, like "red". The documentation implies that START and END are optional, and that the function will apply to the selected region if there is one. But in practice, if you don't specify START and END, nothing happens, so you have to specify them. (region-beginning) and (region-end) work if you have a selected region.

Similarly, I learned that Face->italic from that same menu calls (facemenu-set-italic), and likewise for bold, underline etc. They work on the selected region.

But what if there's no region defined? I decided it might be nice to be able to set styles for the current line, without selecting it first. I can use (line-beginning-position) and (line-end-position) for START and END. So I wrote a wrapper function. For that, I didn't want to use specific functions like (facemenu-set-italic); I wanted to be able pass a property like "italic" to my wrapper function.

I found a way to do that: (put-text-property START END 'italic). But that wasn't quite enough, because put-text-property replaces all properties; you can't make something both italic and bold. To add a property without removing existing ones, use (add-text-properties START END (list 'face 'italic)).

So here's the final code that I put in my .emacs. I was out of excuses to procrastinate, and my enriched-mode bindings worked fine for taking notes on the project which had led to all this procrastination.

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Text colors/styles. You can use this in conjunction with enriched-mode.
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

;; rich-style will affect the style of either the selected region,
;; or the current line if no region is selected.
;; style may be an atom indicating a rich-style face,
;; e.g. 'italic or 'bold, using
;;   (put-text-property START END PROPERTY VALUE &optional OBJECT)
;; or a color string, e.g. "red", using
;;   (facemenu-set-foreground COLOR &optional START END)
;; or nil, in which case style will be removed.
(defun rich-style (style)
  (let* ((start (if (use-region-p)
                    (region-beginning) (line-beginning-position)))
                    
         (end   (if (use-region-p)
                    (region-end)  (line-end-position))))
    (cond
     ((null style)      (set-text-properties start end nil))
     ((stringp style)   (facemenu-set-foreground style start end))
     (t                 (add-text-properties start end (list 'face style)))
     )))

(defun enriched-mode-keys ()
  (define-key enriched-mode-map "\C-ci"
    (lambda () (interactive)    (rich-style 'italic)))
  (define-key enriched-mode-map "\C-cB"
    (lambda () (interactive)    (rich-style 'bold)))
  (define-key enriched-mode-map "\C-cu"
    (lambda () (interactive)    (rich-style 'underline)))
  (define-key enriched-mode-map "\C-cr"
    (lambda () (interactive)    (rich-style "red")))
  ;; Repeat for any other colors you want from rgb.txt

  (define-key enriched-mode-map (kbd "C-c ")
    (lambda () (interactive)    (rich-style nil)))
  )
(add-hook 'enriched-mode-hook 'enriched-mode-keys)

Tags: editors, emacs
[ 14:48 Dec 12, 2015    More linux/editors | permalink to this entry | ]

Global key bindings in Emacs

Global key bindings in emacs. What's hard about that, right? Just something simple like

(global-set-key "\C-m" 'newline-and-indent)

and you're all set.

Well, no. global-set-key gives you a nice key binding that works ... until the next time you load a mode that wants to redefine that key binding out from under you.

For many years I've had a huge collection of mode hooks that run when specific modes load. For instance, python-mode defines \C-c\C-r, my binding that normally runs revert-buffer, to do something called run-python. I never need to run python inside emacs -- I do that in a shell window. But I fairly frequently want to revert a python file back to the last version I saved. So I had a hook that ran whenever python-mode loaded to override that key binding and set it back to what I'd already set it to:

(defun reset-revert-buffer ()
  (define-key python-mode-map "\C-c\C-r" 'revert-buffer) )
(setq python-mode-hook 'reset-revert-buffer)

That worked fine -- but you have to do it for every mode that overrides key bindings and every binding that gets overridden. It's a constant chase, where you keep needing to stop editing whatever you wanted to edit and go add yet another mode-hook to .emacs after chasing down which mode is causing the problem. There must be a better solution.

A web search quickly led me to the StackOverflow discussion Globally override key bindings. I tried the techniques there; but they didn't work.

It took a lot of help from the kind folks on #emacs, but after an hour or so they finally found the key: emulation-mode-map-alists. It's only barely documented -- the key there is "The “active” keymaps in each alist are used before minor-mode-map-alist and minor-mode-overriding-map-alist" -- and there seem to be no examples anywhere on the web for how to use it. It's a list of alists mapping names to keymaps. Oh, clears it right up! Right?

Okay, here's what it means. First you define a new keymap and add your bindings to it:

(defvar global-keys-minor-mode-map (make-sparse-keymap)
  "global-keys-minor-mode keymap.")

(define-key global-keys-minor-mode-map "\C-c\C-r" 'revert-buffer)
(define-key global-keys-minor-mode-map (kbd "C-;") 'insert-date)

Now define a minor mode that will use that keymap. You'll use that minor mode for basically everything.

(define-minor-mode global-keys-minor-mode
  "A minor mode so that global key settings override annoying major modes."
  t "global-keys" 'global-keys-minor-mode-map)

(global-keys-minor-mode 1)

Now build an alist consisting of a list containing a single dotted pair: the name of the minor mode and the keymap.

;; A keymap that's supposed to be consulted before the first
;; minor-mode-map-alist.
(defconst global-minor-mode-alist (list (cons 'global-keys-minor-mode
                                              global-keys-minor-mode-map)))

Finally, set emulation-mode-map-alists to a list containing only the global-minor-mode-alist.

(setf emulation-mode-map-alists '(global-minor-mode-alist))

There's one final step. Even though you want these bindings to be global and work everywhere, there is one place where you might not want them: the minibuffer. To be honest, I'm not sure if this part is necessary, but it sounds like a good idea so I've kept it.

(defun my-minibuffer-setup-hook ()
  (global-keys-minor-mode 0))
(add-hook 'minibuffer-setup-hook 'my-minibuffer-setup-hook)

Whew! It's a lot of work, but it'll let me clean up my .emacs file and save me from endlessly adding new mode-hooks.

Tags: editors, emacs, linux
[ 16:46 Sep 14, 2014    More linux/editors | permalink to this entry | ]

Vim: Set wrapping and indentation according to file type

Although I use emacs for most of my coding, I use vim quite a lot too, for quick edits, mail messages, and anything I need to edit when logged onto a remote server. In particular, that means editing my procmail spam filter files on the mail server.

The spam rules are mostly lists of regular expression patterns, and they can include long lines, such as:
gift ?card .*(Visa|Walgreen|Applebee|Costco|Starbucks|Whitestrips|free|Wal.?mart|Arby)

My default vim settings for editing text, including line wrap, don't work if get a flood of messages offering McDonald's gift cards and decide I need to add a "|McDonald" on the end of that long line.

Of course, I can type ":set tw=0" to turn off wrapping, but who wants to have to do that every time? Surely vim has a way to adjust settings based on file type or location, like emacs has.

It didn't take long to find an example of Project specific settings on the vim wiki. Thank goodness for the example -- I definitely wouldn't have figured that syntax out just from reading manuals. From there, it was easy to make a few modifications and set textwidth=0 if I'm opening a file in my procmail directory:

" Set wrapping/textwidth according to file location and type
function! SetupEnvironment()
  let l:path = expand('%:p')
  if l:path =~ '/home/akkana/Procmail'
    " When editing spam filters, disable wrapping:
    setlocal textwidth=0
endfunction
autocmd! BufReadPost,BufNewFile * call SetupEnvironment()

Nice! But then I remembered other cases where I want to turn off wrapping. For instance, editing source code in cases where emacs doesn't work so well -- like remote logins over slow connections, or machines where emacs isn't even installed, or when I need to do a lot of global substitutes or repetitive operations. So I'd like to be able to turn off wrapping for source code.

I couldn't find any way to just say "all source code file types" in vim. But I can list the ones I use most often. While I was at it, I threw in a special wrap setting for mail files:

" Set wrapping/textwidth according to file location and type
function! SetupEnvironment()
  let l:path = expand('%:p')
  if l:path =~ '/home/akkana/Procmail'
    " When editing spam filters, disable wrapping:
    setlocal textwidth=0
  elseif (&ft == 'python' || &ft == 'c' || &ft == 'html' || &ft == 'php')
    setlocal textwidth=0
  elseif (&ft == 'mail')
    " Slightly narrower width for mail (and override mutt's override):
    setlocal textwidth=68
  else
    " default textwidth slightly narrower than the default
    setlocal textwidth=70
  endif
endfunction
autocmd! BufReadPost,BufNewFile * call SetupEnvironment()

As long as we're looking at language-specific settings, what about doing language-specific indentation like emacs does? I've always suspected vim must have a way to do that, but it doesn't enable it automatically like emacs does. You need to set three variables, assuming you prefer to use spaces rather than tabs:

" Indent specifically for the current filetype
filetype indent on
" Set indent level to 4, using spaces, not tabs
set expandtab shiftwidth=4

Then you can also use useful commands like << and >> for in- and out-denting blocks of code, or ==, for indenting to the right level. It turns out vim's language indenting isn't all that smart, at least for Python, and gets the wrong answer a lot of them time. You can't rely on it as a syntax checker the way you can with emacs. But it's a lot better than no language-specific indentation.

I will be a much happier vimmer now!

Tags: editors, vim, programming
[ 11:29 Jun 15, 2014    More linux/editors | permalink to this entry | ]

The evil HTML double-dash problem in Emacs is still there

Long ago (in 2006!), I blogged on an annoying misfeature of Emacs when editing HTML files: you can't type double dashes. Emacs sees them as an SGML comment and insists on indenting all subsequent lines in strange ways.

I wrote about finding a fix for the problem, involving commenting out four lines in sgml-mode.el. That file had a comment at the very beginning suggesting that they know about the problem and had guarded against it, but obviously it didn't work and the variable that was supposed to control the behavior had been overridden by other hardwired behaviors.

That fix has worked well for eight years. But just lately, I've been getting a lot of annoying warnings when I edit HTML files: "Error: autoloading failed to define function sgml_lexical_context". Apparently the ancient copy of sgml-mode.el that I'd been using all these years was no longer compatible with ... something else somewhere inside emacs. I needed to update it.

Maybe, some time during the intervening 8 years, they'd actually fixed the problem? I was hopeful. I moved my old patched sgml-mode.el aside and edited some files. But the first time I tried typing a double dashes -- like this, with text inside that's long enough to wrap to a new line -- I saw that the problem wasn't fixed at all.

I got a copy of the latest sgml-mode.el -- on Debian, that meant:

apt-get install emacs23-el
cp /usr/share/emacs/23.4/lisp/textmodes/sgml-mode.el.gz ~/.emacs-lisp
gunzip ~/.emacs-lisp/sgml-mode.el.gz

Then I edited the file and started searching for strings like font-lock and comment.

Unfortunately, the solution I documented in my old blog post is no longer helpful. The code has changed too much, and now there are many, many different places where automatic comment handling happens. I had to comment out each of them bit by bit before I finally found the section that's now causing the problem. Commenting out these lines fixed it:

   (set (make-local-variable 'indent-line-function) 'sgml-indent-line)
   (set (make-local-variable 'comment-start) "")
   (set (make-local-variable 'comment-indent-function) 'sgml-comment-indent)
   (set (make-local-variable 'comment-line-break-function)
        'sgml-comment-indent-new-line)

I didn't have to remove any .elc files, like I did in 2006; just putting the sgml-mode.el file in my Emacs load-path was enough. I keep all my customized Emacs code in a directory called .emacs-lisp, and in my .emacs I make sure it's in my path:

(setq load-path (cons "~/.emacs-lisp/" load-path))

And now I can type double dashes again. Whew!

Tags: emacs, editors, html
[ 12:42 Apr 29, 2014    More linux/editors | permalink to this entry | ]

Stop Emacs from invoking a browser

After upgrading my OS (in this case, to Debian sid), I noticed that my browser window kept being replaced with an HTML file I was editing in emacs. I'd hit Back or close the tab, and the next time I checked, there it was again, my HTML source.

I'm sure it's a nice feature that emacs can show me my HTML in a browser. But it's not cool to be replacing my current page without asking. How do I turn it off? A little searching revealed that this was html-autoview-mode, which apparently at some point started defaulting to ON instead of OFF. Running M-x html-autoview-mode toggles it back off for the current session -- but that's no help if I want it off every time I start emacs.

I couldn't find any documentation for this, and the obvious (html-autoview-mode nil) in .emacs didn't work -- first, it gives a syntax error because the function isn't defined until after you've loaded html-mode, but even if you put it in your html-mode hook, it still doesn't work.

I had to read the source of sgml-mode.el. (M-x describe-function html-autoview-mode also would have told me, if I had already loaded html-mode, but I didn't realize that until later.) Turns out html-autoview-mode turns off if its argument is negative, not nil. So I added it to my html derived mode:

(define-derived-mode html-wrap-mode html-mode "HTML wrap mode"
  (auto-fill-mode)
  ;; Don't call an external browser every time you save an html file:
  (html-autoview-mode -1)
)

Tags: emacs, editors, html
[ 22:48 Jun 05, 2013    More linux/editors | permalink to this entry | ]

Converting C to Python with a vi regexp

I'm fiddling with a serial motor controller board, trying to get it working with a Raspberry Pi. (It works nicely with an Arduino, but one thing I'm learning is that everything hardware-related is far easier with Arduino than with RPi.)

The excellent Arduino library helpfully provided by Pololu has a list of all the commands the board understands. Since it's Arduino, they're in C++, and look something like this:

#define QIK_GET_FIRMWARE_VERSION         0x81
#define QIK_GET_ERROR_BYTE               0x82
#define QIK_GET_CONFIGURATION_PARAMETER  0x83
[ ... ]
#define QIK_CONFIG_DEVICE_ID                        0
#define QIK_CONFIG_PWM_PARAMETER                    1

and so on.

On the Arduino side, I'd prefer to use Python, so I need to get them to look more like:

    QIK_GET_FIRMWARE_VERSION = 0x81
    QIK_GET_ERROR_BYTE = 0x82
    QIK_GET_CONFIGURATION_PARAMETER = 0x83
[ ... ]
    QIK_CONFIG_DEVICE_ID = 0
    QIK_CONFIG_PWM_PARAMETER = 1

... and so on ... with an indent at the beginning of each line since I want this to be part of a class.

There are 32 #defines, so of course, I didn't want to make all those changes by hand. So I used vim. It took a little fiddling -- mostly because I'd forgotten that vim doesn't offer + to mean "one or more repetitions", so I had to use * instead. Here's the expression I ended up with:

.,$s/\#define *\([A-Z0-9_]*\) *\(.*\)/ \1 = \2/

In English, you can read this as:

From the current line to the end of the file (,.$/), look for a pattern consisting of only capital letters, digits and underscores ([A-Z0-9_]). Save that as expression #1 (\( \)). Skip over any spaces, then take the rest of the line (.*), and call it expression #2 (\( \)).

Then replace all that with a new line consisting of 4 spaces, expression 1, a spaced-out equals sign, and expression 2 ( \1 = \2).

Who knew that all you needed was a one-line regular expression to translate C into Python?

(Okay, so maybe it's not quite that simple. Too bad a regexp won't handle the logic inside the library as well, and the pin assignments.)

Tags: regexp, linux, programming, python, editors, vim
[ 21:38 Jan 19, 2013    More linux/editors | permalink to this entry | ]

Integrating graphics with text in Emacs

I discussed Emacs's artist-mode a few days ago as a simple, but incomplete, solution to the problem of sketching graphs while taking notes during a math class. But I've found a much better way, one that allows for including any images -- drawings, photos, or screenshots. It took a little work and some custom .emacs code, but I love the result.

Iimage mode

The key is iimage-mode, which displays inline images. In this mode, you put a line in your buffer with a reference to your image file, something like this:

file://myimage.jpg

and Emacs will replace it with the contents of that image. Marvellous!

You can use other patterns for filenames as well, but I'm fine with using URLs. Note there are only two slashes in file:// -- it's a local file in the same directory as the text file being edited.

It's a little tricky to enable it. The docs are not entirely clear on the differences between iimage-mode, turn-on-iimage-mode and iimage-mode-buffer. I found I could get a file that already had existing images to display them with:

  (turn-on-iimage-mode)
  (iimage-mode-buffer t)

Very cool! But too much to type every time. And to use it for note-taking, I needed a way to say, "Create a new image here, let me edit it, then display the image I just edited inline."

Enabling iimage-mode automatically

First, I wanted iimage mode displayed automatically on files in my note-taking directories. I normally use text-mode for these files, with spell checking and line wrapping turned on (auto-fill mode). So I defined a new minor mode based on text-mode:

(define-derived-mode text-img-mode text-mode "Image display mode"
  (auto-fill-mode)
  (turn-on-iimage-mode)
  (iimage-mode-buffer t)
  )

Then I wanted this mode to be called whenever I'm editing a file in my classes directory. So I added it to my auto-mode-alist:

(setq auto-mode-alist
   ...
      (cons '("Docs/classes/" . text-img-mode)
   ...
      auto-mode-alist) )

Inserting a new image

Next, I needed a way to insert an image URL into the buffer and call up an image editor on it. I shouldn't have to type the filename twice and keep track of it; that's what computers are for.

And I needed a drawing program. As a longtime GIMP geek, most of my computer drawing has been in GIMP. But GIMP is overkill for calling up a quick sketch window. I was tempted to use TuxPaint; it's a good sketching app even if you're not five years old, and it's fun and easy to use. But by default, TuxPaint has some features that get in the way of note-taking, like distracting sound effects. I'm sure it's possible to turn those off, and I do plan to investigate that.

I saw a reference to pinta as a lightweight drawing app, but it required a boatload of Mono libraries that I don't otherwise need, and Krita has the same problem with KDE services. So I opted for MyPaint. It works okay, though it's rather slow to start up and has some other issues, so I'm still hoping to find a more lightweight sketching app.

In any case, I fiddled around with start-process until I figured out how to use it to start a program. Then I wrote a little function that lets the user pick a filename, inserts a URL to that filename into the buffer, then calls up mypaint on the file.

(defun img ()
  "Prompt for a filename, then call up mypaint to create an image"
  (interactive)
  (let ((imgfile (read-string "Filename? " "xxx.jpg" 'my-history)))
    (insert "\nfile://" imgfile "\n" )
    (start-process "mypaint" nil "/usr/bin/mypaint" imgfile)
  ))

Worked fine! I can run M-x img, be prompted for a filename, and get a mypaint window where I can make my sketch.

Noticing that a new image has been added

But wait. I finish sketching, write the file and quit mypaint ... and the buffer still shows something like file://xxx.jpg, even if it's showing other images inline. I needed a way to tell it to refresh and load any new images. (I considered having emacs wait for mypaint to exit, but decided I might sometimes want to keep editing while mypaint was still up.)

M-x eval-expression (iimage-mode-buffer t) will do that, but that's a lot of typing to do. Obviously, I needed a key binding.

Strangely enough, C-c i wasn't taken for text buffers, so that seemed like a natural. So I added a key binding to the end of the text-img-mode. iimage-mode-buffer requires that t argument -- it gives an error without it -- so the key binding looks a little more complicated than one that just calls a simple function. I added it to the end of my text-img-mode function.

(define-derived-mode text-img-mode text-mode "Image display mode"
 ...
  (local-set-key "\C-ci" 
    (lambda () (interactive) (iimage-mode-buffer t)))
  )

But after using it a bit, I discovered that this didn't reload images if I edited them a second time. Fortunately, vwood had the answer:

(defun refresh-iimages ()
  "Only way I've found to refresh iimages (without also recentering)"
  (interactive)
  (clear-image-cache nil)
  (iimage-mode nil)
  (iimage-mode t)
  (message "Refreshed images")
 )

I added the message at the end, since otherwise the function left a distracting "Toggling iimage-mode off; better pass an explicit argument" error.

Then the key binding in my text-img-mode became

(local-set-key "\C-ci" 'refresh-iimages)

Inserting a screenshot

Wait -- one more thing. As I actually used text-img-mode to take notes, I discovered that taking screenshots would actually be much more useful than making my own drawings. Then I could copy small sections of the slides and graphs into my notes at the appropriate place, without needing to copy equations at all.

Why not write a function to allow that? The unpleasantly named scrot program fills the bill nicely, and gives me a choice of clicking in a window or dragging out an area of the screen.

(defun screenshot ()
 "Prompt for a filename, then call up scrot to create an interactive screenshot"
  (interactive)
  (let ((imgfile (read-string "Filename? " "scr.jpg" 'my-history)))
    (insert "\nfile://" imgfile "\n" )
    (start-process "scrot" nil "/usr/bin/scrot" "-s" imgfile)
  ))

This turned out to be so useful that I added a key for it in text-img-mode:

(local-set-key "\C-cs" 'screenshot)

I'm so happy with the result! Iimage mode is working great, and having text and images together is turning out to be perfect for note-taking.

My only problem now -- okay, I admit it -- is a tendency to get so excited over inserting screenshots that I get distracted and forget to actually listen to the lecture. I'm sure I'll get over that, but for now, Thank goodness vlc is good at skipping back!

Tags: editors, emacs, graphics, education
[ 13:42 Jan 12, 2013    More linux/editors | permalink to this entry | ]

ASCII graphics in Emacs with Artist Mode

I found a cool package in Emacs the other day: Artist Mode. It lets you draw ASCII graphics in your text file.

I was actually looking for the solution to a different problem: taking notes in a math-intensive Coursera class. I've been taking notes in Emacs, but a text editor is awkward for equations and even more awkward for graphs.

What I really wanted was something like the old Claris Works (or so I'm told; I never used it myself) -- something that's primarily a text editor but lets you drawings, equations, and tables when you need to. In theory, word processors like LibreOffice could do that, but in practice they're not very good at switching modes, nor at integrating several types of media into one document. Texmacs is great for the equations and apparently it can do tables too, but it can't do freehand drawing.

And none of these programs is very configurable -- I can't use my fast, comfortable Emacs bindings while typing, and that's a deal-breaker for me, because being able to make corrections quickly makes a huge difference in my typing speed. LibreOffice's key bindings are only partially configurable, and after you've spent half a day chasing down all the action names you need (the ones that are actually available), you upgrade to a newer version and discover you have to do it all over again because there's no way to migrate configuration files. Even Texmacs, ironically, is no better: the documentation claims it's possible to configure key bindings, but it doesn't appear anyone has ever succeeded in figuring out how.

Anyway, ASCII graphics aren't the ultimate solution to note-taking. And I've found a better solution for that, while I'll write about separately. But for now, Artist Mode is just so cool I had to share it.

Enable it by running M-x artist-mode. You can immediately start drawing in your buffer with the mouse. Whatever you draw gets turned into ASCII graphics.

For note-taking, it's fine for scribbling the rough shape of a curve. It takes no time to mouse in a little sketch like

       |                               .. 20
       |                              ..
       |                            ...
    10 |.                          ..
       |..                       ...
       | ...                   ...
       |   ..              ....
       |    .....      .....
       |    ............
       |.....        .............
       +-------------------------------------

It even has primitives (middleclick to get a.menu) for things like lines, rectangles and circles, and for filling regions. When you're done drawing, M-x artist-mode goes back to whatever mode you were using before.

I probably won't use it very much for note taking. But there are times when I've wanted to draw ASCII graphics -- a laborious process in ordinary text modes -- and other times when it would just be fun to play around with my buffer. I'm happy to know about Artist Mode. I may not need it often, but it sure is fun to use now and then.

Tags: editors, emacs, graphics, education
[ 20:02 Jan 10, 2013    More linux/editors | permalink to this entry | ]

Customize the Emacs modeline color

I wrote last week about how to customize syntax highlighting colors in Emacs. And as part of that, I ditched the color theme I'd been using and let Emacs go back to its default colors.

Which mostly was fine, except that when I split the window into two windows, to look at two files at once or two different parts of the same file, the separator between the two windows -- the mode line -- was the same grey as Emacs's normal background, so it wasn't very obvious where the window split was.

A web search turned out lots of different ways to set the mode line color. Many of them involve color themes and are fairly complicated. Here's the simplest method I found:

(set-face-foreground 'modeline "white")
(set-face-background 'modeline "purple")
(set-face-background 'modeline-inactive "light blue")

You can set your active mode line to a pretty color, so it stands out a bit and makes it easy to tell which of the visible windows is the one you're actually typing in, and set the inactive mode lines -- windows that are visible but you arne't actually typing in -- to a less striking color.

Tags: editors, emacs
[ 13:50 Jan 02, 2013    More linux/editors | permalink to this entry | ]

Customizing syntax highlighting colors in Emacs

Emacs has wonderful syntax highlighting. Words will be displayed in different colors depending on their syntax and the mode of the current file -- for instance, in C code, keywords of the language are highlighted in one color, comments in another, strings in a third.

The problem comes when the colors aren't right. Like that awful gold color that the flyspell spell checker uses for some words. Against a light background it makes the words almost impossible to read. I've struggled for years trying to set up custom color schemes to get around that problem, but I finally learned a simpler way to handle it when you see something in a color you want to change.

The trick is to find out what face you need to change. "Face" to emacs means more than a font face like Sans or Lucida; it means a collection of information about how characters are displayed, including font face, weight, slant, color and other attributes.

When you see something displayed in a color you don't want, place the cursor somewhere in the word. type C-u C-x = will get you the face used, along with all sorts of information about it and a handy Customize what to show link. Or you can go straight to the Customize screen with M-x customize-face -- hit return to customize the face at point (the cursor location).

In the customize-face screen, there's no GUI to choose colors, but you can edit color names. Emacs lists "red" as the foreground color; if you change it to "blue" you'll see a preview of how it will look. Color names come from /etc/X11/rgb.txt, and there are various programs like xcolorsel that will show them -- or better yet, see Wikipedia's X11 color names chart.

Once you've chosen a color, the Save for future sessions button will add a section to your .emacs file with the appropriate elisp code. Of course, you can move this code elsewhere as well. I have a somewhat complex .emacs setup, so I've moved the code into another file. Strangely, I found that my .Xdefaults background color setting no longer worked once I started using custom-set-faces, so I added a line for that as well.

(set-background-color "grey90")
(custom-set-faces
 '(flyspell-duplicate ((((class color)) (:foreground "red" :underline t :weight bold))))
 '(font-lock-comment-face ((((class color) (min-colors 88) (background light)) (:foreground "blue"))))
 )

Tags: emacs, editors
[ 13:59 Dec 23, 2012    More linux/editors | permalink to this entry | ]

Vim tip: opening lines without delay

I use vim to reply to email messages in mutt. And something I do very often is to open some new lines between quoted sections. For instance, if I'm replying to mail with a section like

> Have you noticed anything weird about squirrels lately?
> I think they're plotting something.

I might want to open up space after the first line, and say

> Have you noticed anything weird about squirrels lately?

You too? One of our local squirrels ran off with my Arduino yesterday.

> I think they're plotting something.

So I need three blank lines, with the cursor on the middle line.

In vim, typically I would place the cursor on the line before the space I want to open, and type o<RETURN><ESC>O :

o

open a line below the current line

<RETURN>

make an empty line below the quoted line

<ESC>

get out of insert mode

O

open a new line above the current line and enter insert mode there

Sounds complicated, but it's pretty fast to type o<RET><ESC>O especially when you do it many times a day.

Except vim has an odd quirk: after you type that O, you have to wait a second or two. If you start typing immediately, somehow you're not in insert mode any more, and the line you just opened, and thought you were typing in, disappears.

Well, it turns out this happens because <ESC>O is a sequence that can also happen from arrow keys. So vim has a timeout it uses to figure out whether an <ESC>O sequence was from an arrow key, or two characters typed by a real person. If the O follows too soon after the <ESC>, it figures it's an arrow key. (This is discussed in more detail in this Stack Overflow thread: Vim: Delay before 'O' opens a new line?)

It turns out there's a simple solution: set the timeout much shorter, like 100 milliseconds, in .vimrc:

set ttimeoutlen=100

Then arrow keys should still work -- unless you typically use vim over a very slow connection -- but you can type the sequence yourself (unless you type very fast) and not have it confused with an arrow key.

But really, there's a better solution. Since opening up space like this is something I do so often, why not make a vim binding for it?

map <leader>o o<CR><ESC>O

Now all I have to do is type \o (backslash, \, is the "leader" character) and I get a space opened up, with a blank line both before it and after it.

Tags: editors, vim
[ 12:20 Jul 21, 2012    More linux/editors | permalink to this entry | ]

Editing tab-separated files with Emacs

I wanted to keep maintenance records for my car in a local file. Things like the date and mileage of the last oil change, when the timing belt was replaced, stuff like that.

I didn't want to hassle with databases or spreadsheets -- a simple, human-readable file is a lot easier to deal with. But I also wanted it in a predictable format so it could potentially be parsed later: I might some day want to write a program that keeps track and reminds me when I need to do things. (You'd think this program would already exist, but curiously, I haven't found one.)

So, something like:

7/9/12 <TAB> 86306 <TAB> Oil change, filter
6/11/12<TAB> 84813 <TAB> Smog test

Simple, right? And super easy just to type in in a text editor.

Well ... in emacs, maybe not. As an editor oriented toward programmers, emacs tends to prefer spaces instead of tabs ... and that's normally the way I prefer it. Tabs are a bad idea in most software projects, unless everybody on the project has already agreed on a tab style and width. (The complete tabs-vs.spaces religious war is beyond the scope of this article, but take my word for it that they can be a problem.)

But this wasn't code, and I needed an easy way to get those tabs into the file. Of course, I could quote them directly: type Ctrl-Q TAB every time I wanted a tab. But I knew I wouldn't remember to do that every time.

So I needed a local variables line, sometimes called a modeline, to flag this file as different from any other file I edit -- in this one, I really do want a TAB when I hit the TAB key.

My first try put this as the first line of the file:

-*- Mode: Text; indent-tabs-mode: t -*-

(Actually, my first try omitted the -*- part and didn't work at all, but let's ignore that part.)

When I re-loaded the file, emacs loaded it in text mode, and indent-tabs-mode was set to t if I checked it with describe-variable. Great! I could add new lines at the end of the file, and when I hit TAB, emacs inserted tabs. But wait -- if I added a line at the beginning of the file, and typed something lke 7/9/12<TAB> ... it inserted a space, not a tab.

It turns out indent-tabs-mode isn't really documented. If you check the help, it says:

Indentation can insert tabs if this is non-nil

-- note that "can" insert, not "will" insert, a tab. So how could I get emacs to really insert a tab, every time I hit the tab key?

Well, of course, I could bind the TAB key to "self-insert". But I didn't want to change it for every file, only for this one. how could I make that happen as part of a local variables line? The local variables line documentation shows how to set variables or file modes, but it looks like you can't bind keys or run functions.

Well, you can't run functions, but you can define a new mode that runs functions. At the suggestion of some helpful people on the #emacs IRC channel, I defined a new mode in my .emacs, like this:

(define-derived-mode tabbed-mode text-mode "Tab separated mode"
  (local-set-key (kbd "TAB") 'self-insert-command)
  )

Then in the file, I put this line:

-*- Mode: Tabbed -*-

Now, every time I load the file, emacs turns on tabbed mode -- which is just text mode except that the TAB key always inserts a tab.

Tags: editors, emacs
[ 19:35 Jul 16, 2012    More linux/editors | permalink to this entry | ]

Using default-frame-alist to specify Emacs window size, position and font

Seems like every few years I need to change the way I specify my preferred emacs fonts and window sizes.

Historically this all used to happen from one file, ~/.Xdefaults, where you set up your defaults for all X programs. In a way that was nice, since you could set up defaults and see the same font everywhere. On the other hand, it made for a huge, somewhat hard to read file, and it's increasingly out of favor on modern desktops, with modern toolkits like GTK just ignoring it.

Emacs still reads Xdefaults -- but only sort of. A lot of the values I used to set there no longer work properly. Some time ago I commented out my various attempts at setting emacs font, like

Emacs*font: -*-clean-bold-*-*-*-13-*-*-*-c-*-*-*
Emacs*font: DejaVu Sans Mono-10:bold
Emacs*font: clean-13:bold
Wmacs*font: Liberation Mono-10:bold
Emacs.font: 7x13bold
Emacs.faceName: Dejavu-10:style=bold

since none of them worked, and worked out a way of setting fonts inside my .emacs file:

(set-face-attribute 'default nil :font "Terminus-12:bold")

That worked to set the font, but it had another annoying attribute: it doesn't happen at startup, so it messed up my window size. See, emacs would start up, see the size I specified in .Xdefaults:

Emacs*geometry: 80x45

and try to set that. But it hadn't read .emacs yet, so it was still using whatever its default font and size is, and that's huge -- so 45 lines made a window too tall to fit on my laptop screen. Emacs would then shrink its window to fit the screen (41 lines). Only then would it open .emacs, whereupon it would see the set-face-attribute, change the font, and resize the window again, much, smaller, still 41 lines.

What a pain!

The emacs manual, in addition to talking about these various Xdefaults properties and command-line options, does mention a couple of variables, set-screen-height and set-screen-width, that looked promising. I tried putting (set-screen-height 45) in my .emacs right after I set the font -- no dice. Apparently that doesn't work because by the time those are read, emacs has already decided that 41 lines is as big as the window can possibly be.

Here's the answer: another variable that goes inside .emacs, default-frame-alist, but this one can override that maximum-height decision that emacs has already made. Here's an example of it in some useful defaults for emacs, and based on that, I was able to come up with this:

(setq default-frame-alist
      '((top . 10) (left . 2)
        (width . 80) (height . 53)
        (font . "terminus-iso8859-1-bold-14")
        ))

Curiously, that height setting, 53, needs to be 3 more than what I actually want according to the size emacs reports to the window manager. So don't take the number too seriously; just try numbers a little bigger than what you actually want until you get the size you're after. The font setting is the X font specifier: I ran xlsfonts | grep -i terminus | grep 14 then picked one of the simpler of the lines it printed out, but you can use a full specifier like -xos4-terminus-bold-r-normal--14-140-72-72-c-80-iso8859-1 like you get from xfontsel, if you prefer.

Startup still isn't pretty -- emacs still shows a big window at one place on the screen, resizes it several times then jumps it over to the top/left coordinates I specified. Of course, I could tell my window manager to start it in the right place so the jumping-around would be minimized; but that wouldn't help the visible resizing. Just a minor irritation.

I'm sure there's lots more useful stuff buried in that sample emacs config file (it was suggested to me when I asked about this on the #emacs IRC channel), so I'll be reading it to see what else I can glean.

Tags: emacs, editors, X11
[ 12:24 Jul 14, 2011    More linux/editors | permalink to this entry | ]

Vim/Emacs tip: use modelines for files that need special behavior

Most of the time when I edit a text file in vim, I want lines to wrap automatically as I type, at somewhere around 70 columns wide. So I set textwidth=70 in .vimrc.

But sometimes that isn't appropriate. For instance, I have a procmail rules file where I put common spam patterns that programs like spamassassin don't seem to be able to catch. So I might have lines like:

*^Subject:.*(Ink Cartridges|Hummingbird Vine|who's who of executives|Avandia|Botox|Your Email ID|Zoosk|Best airfares on the internet|UGG Boots|police training)

... and so on -- you get the idea. I can't have lines breaking in the middle, because then the procmail rule wouldn't work. So every time I add a new phrase, I have to :set tw=0 (or one of the other umpteen ways one can tell vim not to wrap lines) first.

But you can set special behavior for one specific file by adding a special comment called a "modeline" as the first line of the file.

Procmail treats any line starting with a hash, #, as a comment, and vim recognizes # as a comment. So I can add this as the first line of the procmail file:

# vim: set tw=0:

then vim will see that and un-set that default text width I specify in .vimrc.

Vim understands most common comment styles, so it should understand lines like /* vim: set tw=0: */ and // vim: set tw=0: and ; vim: set tw=0: as well.

But to make this work I had to do one more thing: in .vimrc, I had to add

set modeline

Apparently on some versions of vim this is on by default; in others it's turned off for security reasons (someone could put an evil modeline into a file which would make your vim do something when you edited it). Definitely something to be aware of, but if you mostly edit files you created yourself on your local machine, and no one else uses your machine, it's your choice whether to worry about it.

Emacs has modelines too

Emacs has mode lines too, though it calls them Local variables lines. For instance, C++ files in Mozilla's source tend to start with:

/* -*- Mode: C++; tab-width: 2; indent-tabs-mode: nil; c-basic-offset: 2 -*- */

It's awfully handy to be able to define specific indentation style for the files within a project, making it easy for emacs users, at least, to follow your preferred coding style. If only all editors understood them!

Tags: editors, vim, emacs, tip
[ 21:26 May 25, 2011    More linux/editors | permalink to this entry | ]

Quick tip: Disabling version control in Emacs

For some time I've been mildly annoyed that whenever I start emacs and open a file that's under any sort of version control -- cvs, svn, git or whatever -- I can't start editing right away, because emacs has to pause for a while and load a bunch of version-control cruft I never use. Sometimes it also causes problems later, when I try to write to the file or if I update the directory.

It wasn't obvious what keywords to search for, but I finally found a combination, emacs prevent OR disable autoload vc (the vc was the important part), which led me to the solution (found on this page):

;; Disable all version control
(setq vc-handled-backends nil)

Files load much faster now!

Tags: editors, emacs, git, tip
[ 13:11 Feb 04, 2011    More linux/editors | permalink to this entry | ]

Escaping HTML characters in Emacs (and how to do replaces in elisp)

Last week I found myself writing another article that includes code snippets in HTML.

So what, you ask? The problem is, when you're writing articles in HTML, every time you include a code snippet inside a <pre> tag you invariably forget that special characters like < > & have special meanings in HTML, and must be escaped. Every < has to change to &lt;, and so forth, after you paste the code.

In vi/vim, replacing characters is straightforward. But I usually write longer articles in emacs, for various unimportant reasons, and although emacs has global replace, it only works from wherever you are now (called "point" in emacs lingo) to the end of the file. So if you're trying to fix something you pasted in the middle of the article, you can't do it with normal emacs replace.

Surely this is a wheel that has already been re-invented a thousand times, I thought! But googling and asking emacs experts turned up nothing. Looks like I'd have to write it.

And that turned out to be more difficult than I expected, for the same reason: emacs replace-string works the same way from a program as it does interactively, and replaces from point to the end of the file, and there's no way to restrict it to a more limited range.

Several helpful people on #emacs chimed in with ideas, but most of them didn't pan out. But ggole knew a way to do it that was both clean and reliable (thanks!).

Here's the elisp function I ended up with. It uses save-excursion to put the cursor back where it started before you ran the function, narrow-to-region to make replace-string work only on the region, and save-restriction get rid of that narrow-to-region after we're done. Nice!

(defun unhtml (start end)
  (interactive "r")
  (save-excursion
    (save-restriction
      (narrow-to-region start end)
      (goto-char (point-min))
      (replace-string "&" "&")
      (goto-char (point-min))
      (replace-string "<" "&lt;")
      (goto-char (point-min))
      (replace-string ">" "&gt;")
      )))

And yes, I used it just now on that elisp snippet.

Tags: emacs, editors, programming
[ 20:08 Dec 01, 2010    More linux/editors | permalink to this entry | ]

Non-greedy regular expressions to clean up crappy autogenerated HTML

I maintain the websites for several clubs. No surprise there -- anyone with even a moderate knowledge of HTML, or just a lack of fear of it, invariably gets drafted into that job in any non-computer club.

In one club, the person in charge of scheduling sends out an elaborate document every three months in various formats -- Word, RTF, Excel, it's different each time. The only regularity is that it's always full of crap that makes it hard to turn it into a nice simple HTML table.

This quarter, the formats were Word and RTF. I used unrtf to turn the RTF version into HTML -- and found a horrifying page full of lines like this:

<body><font size=3></font><font size=3><br>
</font><font size=3></font><b><font size=4></font></b><b><font size=4><table border=2>
</font></b><b><font size=4><tr><td><b><font size=4><font face="Arial">Club Schedule</font></font></b><b><font size=4></font></b><b><font size=4></font></b></td>
<font size=3></font><font size=3><td><font size=3><b><font face="Arial">April 13</font></b></font><font size=3></font><font size=3><br>
</font><font size=3></font><font size=3><b></b></font></td>

I've put the actual page content in bold; the rest is just junk, mostly doing nothing, mostly not even legal HTML, that needs to be eliminated if I want the page to load and display reasonably.

I didn't want to clean up that mess by hand! So I needed some regular expressions to clean it up in an editor. I tried emacs first, but emacs makes it hard to try an expression then modify it a little when the first try doesn't work, so I switched to vim.

The key to this sort of cleanup is non-greedy regular expressions. When you have a bad tag sandwiched in the middle of a line containing other tags, you want to remove everything from the <font through the next > -- but no farther, or else you'll delete real content. If you have a line like

<td><font size=3>Hello<font> world</td>

you only want to delete through the <font>, not through the </td>.

In general, you make a regular expression non-greedy by adding a ? after the wildcard -- e.g. <font.*?>. But that doesn't work in vim. In vim, you have to use \{M,N} which matches from M to N repetitions of whatever immediately precedes it. You can also use the shortcut \{-} to mean the same thing as *? (0 or more matches) in other programs.

Using that, I built up a series of regexp substitutes to clean up that unrtf mess in vim:

:%s/<\/\{0,1}font.\{-}>//g
:%s/<b><\/b>//g
:%s/<\/b><b>//g
:%s/<\/i><i>//g
:%s/<td><\/td>/<td><br><\/td>/g
:%s/<\/\{0,1}span.\{-}>//g
:%s/<\/\{0,1}center>//g

That took care of 90% of the work, leaving me with hardly any cleanup I needed to do by hand. I'll definitely keep that list around for the next time I need to do this.

Tags: regexp, html, editors, vim
[ 23:02 Mar 29, 2010    More linux/editors | permalink to this entry | ]

Make scroll lock stop beeping in Emacs 23

I haven't been using the spare machine much lately. So I hadn't noticed until last week that since upgrading to the emacs 23.1.1 on Ubuntu Karmic koala, every time I press the Scroll Lock key -- the key my KVM uses to switch to the other computer -- with focus in an emacs window, emacs beeps and complains that the key is unbound.

That was a problem I thought I'd solved long ago, an easy fix in .emacs:

(global-set-key [scroll-lock] 'ignore)

But in emacs 23, it wasn't working any more. Emacs listed the key as "<Scroll_Lock>", but using that directly in global-set-key doesn't work.

The friendly and helpful (really!) crew at #emacs found me a solution, after some fiddling around.

(global-set-key (kbd "<Scroll_Lock>") 'ignore)

Tags: emacs, editors, kvm, tips
[ 23:47 Feb 09, 2010    More linux/editors | permalink to this entry | ]

Taming Emacs' text mode wrapping and indenting

To wrap long lines, or not to wrap? It's always a dilemma. Automatic wrapping is great when you're hammering away typing lots of text. But it's infuriating when you're trying to format something yourself and the editor decides it wants to line-wrap a little too early.

Although of course you can set the wrapping width, Emacs has a tendency to wrap early -- especially when you hit return. All too often, I'll be typing away at a long line, get to the end of the sentence and paragraph with the last word on the same line with the rest -- then realize that as soon as I hit return, Emacs is going to move that last word to a line by itself. Drives me nuts!

And the solution turns out to be so simple. The Return key, "\C-m". was bound to the (newline) function (you can find out by typing M-x, then describe-key, then hitting Return). Apparently (newline) re-wraps the current line before inserting a line break. But I just wanted it to insert a line break.

No problem -- just bind "C-m" to (insert "\n").

But there's a second way, too, if you don't want to rebind: there's a magic internal emacs table you can change.

(set-char-table-range auto-fill-chars 10 nil)

But wait -- there's one other thing I want to fix in text mode.

Automatic indent is another one of those features that's very convenient ... except when it's not.

If I have some text like:

First point:
  - subpoint a
  - subpoint b

then it's handy if, when I hit Return after subpoint a, emacs indents to the right level for subpoint b. But what happens when I get to the end of that list?

First point:
  - subpoint a
  - subpoint b

Second point:
  - subpoint c

When I hit Return after subpoint b, Emacs quite reasonably indents two spaces. If I immediately type another Return, Emacs sensibly deletes the two spaces it just inserted, opens a new line -- but then it indents that new line another two spaces.

After a blank line, I always want to start at the beginning, not indented at all.

Here's how to fix that. Define a function that will be called whenever you hit return in text mode. That function tests whether the caret comes immediately after a blank line, or at the beginning of the file. It indents except in those two cases; and in neither case does it re-wrap the current line.

;; In text mode, I don't want it auto-indenting for the first
;; line in the file, or lines following blank lines.
;; Everywhere else is okay.
(defun newline-and-text-indent ()
  "Insert a newline, then indent the next line sensibly for text"
  (interactive)
  (cond
   ;; Beginning of buffer, or beginning of an existing line, don't indent:
   ((or (bobp) (bolp)) (newline))

   ;; If we're on a whitespace-only line,
   ((and (eolp)
         (save-excursion (re-search-backward "^\\(\\s \\)*$"
                                             (line-beginning-position) t)))
    ;; ... delete the whitespace, then add another newline:
    (kill-line 0)
    (newline))

   ;; Else (not on whitespace-only) insert a newline,
   ;; then add the appropriate indent:
   (t (insert "\n")
      (indent-according-to-mode))
   ))

Then tell emacs to call that function when it sees the Return key in text mode:

(defun text-indent-hook ()
  (local-set-key "\C-m" 'newline-and-text-indent)
  )
(setq text-mode-hook 'text-indent-hook)

Finally, this is great for HTML mode too, if you get irritated at not being able to put an <a href="longurl"> all on one line:

(defun html-hook ()
  (local-set-key "\C-m" (lambda () (interactive) (insert "\n")))
  )
(setq sgml-mode-hook 'html-hook)

Tags: emacs, editors, tips
[ 11:29 Jan 13, 2010    More linux/editors | permalink to this entry | ]

Emacs: Insert tags in HTML Mode

Wouldn't it be nice if Emacs HTML mode had a way to insert HTML tags, so you didn't have to type <b></b> all the time? Sort of like what's described in this page -- except that page describes an HTML mode that clearly isn't the one that's installed on Ubuntu, since none of those bindings actually work?

I've been meaning to figure out a way to do that for ages, and finally got around to it. Turns out Emacs SGML mode (which is really what Ubuntu installs and uses for HTML files) doesn't have functions for specific HTML tags like <b>, but it does have a general tag-inserting function.

Type C-c C-t -- emacs prompts you for the tag, so type b or whatever, and hit return -- and you get the tag, with the cursor correctly positioned for you to type your new bold text.

But that's four keystrokes. What if you want shorter bindings for particular tags, like C-b C-b to insert a bold?

For that, you need to use a lambda and a mode hook. In your .emacs it looks like this:

;; Define keys for inserting tags in HTML mode:
(defun html-hook ()
  (local-set-key "\C-c\C-b" (lambda () (interactive) (sgml-tag "b")))
  )
(setq sgml-mode-hook 'html-hook)

There's apparently also supposed to be a command bound to C-c / that closes the current tag, but my version of sgml-mode doesn't bind anything to that key, and the only likely-looking function name, sgml-maybe-end-tag, doesn't end the current tag. Such is life!

But one more don't-miss feature that I'd missed all along is C-c C-n: type it before a special character like < or & and emacs will insert the appropriate &lt; or &amp; for you. Nice!

(Thanks to bojohan on #emacs for the tips!)

Tags: editors, emacs, html-mode
[ 21:31 Jul 29, 2009    More linux/editors | permalink to this entry | ]

Emacs bookmarks -- a huge time-saver

Oh, wow. I can't believe I've used Emacs all these years without knowing about bookmarks.

I wanted something in Emacs akin to the "Open Recent" menu that a lot of GUI apps have. Except, well, I didn't want it to need a menu (I don't normally show a menubar in Emacs) and I didn't want it limited only to recently accessed files. So ... just like Open Recent, only completely different.

What I really wanted was a way to nickname files I access regularly, so I don't have to type ~/foo/bar/blaz/route-66/dufus/velociraptor/archaeopteryx/filename every time. Even with tab completion, remembering long paths gets old. Of course emacs must have a way to do that; it has everything. The trick was guessing what it might be called in order to search for it.

The answer is emacs bookmarks and they're super easy to use.

C-x r m sets a bookmark for the current location in the current file. It prompts for a bookmark name; give it a nickname, or hit return to default it to the current filename.

C-x r b bookmark-name jumps back to a bookmark, opening the file if it isn't already. Of course, tab completion works for the bookmark name.

Bookmarks are saved in ~/.emacs.bmk so they're persistent.

It's perfect. I just wish I'd thought to look for it years ago.

(Of course, Emacs can do recent files too.)

Tags: editors, emacs, bookmarks
[ 10:21 Mar 27, 2009    More linux/editors | permalink to this entry | ]

Vim tip: fixing the light-background color schemes

I use a light background for my X terminals (xterm and rxvt): not white, but a light grey that I find easy on the eyes. Long ago, I spent the time to set up a custom vim color scheme that works with the light background.

But sometimes I need to run vim somewhere where I don't have access to my custom scheme. It always starts up with a lot of the words displayed in yellow, completely unreadable against a light background. :set background=light doesn't help -- the default colorscheme is already intended for a light background, yet it still uses yellow characters.

I tried all the colorschemes installed with ubuntu's vim (you can get a list of them with ls /usr/share/vim/vim71/colors). The only light-background vim schemes that don't use yellow all have their primary text color as red. Do a lot of people really want to edit red text? Maybe the same people who think that yellow is a dark color?

Curiously, it turns out that if you use one of these light color schemes on a Linux console (with its black background), the yellow text isn't yellow (which would show up fine against black), but orange (which would be better on a light background).

Mikael knew the answer:

:set t_Co=88

This tells vim to use 88-color mode instead of its default of 8, and the yellow text turns light blue. Not terrifically readable but much better than yellow. Or, instead, try

:set t_Co=256

and the yellow/light blue text turns an ugly, but readable, orange (probably the same orange as the console used).

So, vim users with dark-on-light terminal schemes: add set t_Co=256 in your .vimrc (no colon) and you'll be much happier.

Update: Pádraig Brady has a great page explaining more about terminal colour highlights, including a TERM=xterm-256color setting to get vim to use 256 colors automatically. There's also a lot of good advice there on enabling colors in other console apps.

The only catch: on Ubuntu you do have to install the ncurses-term package, which will get you xterm-256color as well as 256color variants for lots of other terminal types. Here's useful page on 256-Color XTerms in Ubuntu.

Tags: editors, vim, color, tips
[ 22:29 Mar 22, 2009    More linux/editors | permalink to this entry | ]

More fun with regexps: Adding "[no output]" in shell logs

Someone on LinuxChix' techtalk list asked whether she could get tcsh to print "[no output]" after any command that doesn't produce output, so that when she makes logs to help her co-workers, they will seem clearer.

I don't know of a way to do that in any shell (the shell would have to capture the output of every command; emacs' shell-mode does that but I don't think any real shells do) but it seemed like it ought to be straightforward enough to do as a regular expression substitute in vi. You're looking for lines where a line beginning with a prompt is followed immediately by another line beginning with a prompt; the goal is to insert a new line consisting only of "[no output]" between the two lines.

It turned out to be pretty easy in vim. Here it is:

:%s/\(^% .*$\n\)\(% \)/\1[no results]\r\2/

Explanation:

:

starts a command

%

do the following command on every line of this file

s/

start a global substitute command

\(

start a "capture group" -- you'll see what it does soon

^

match only patterns starting at the beginning of a line

%

look for a % followed by a space (your prompt)

.*

after the prompt, match any other characters until...

$

the end of the line, after which...

\n

there should be a newline character

\)

end the capture group after the newline character

\(

start a second capture group

%

look for another prompt. In other words, this whole

expression will only match when a line starting with a prompt

is followed immediately by another line starting with a prompt.

\)

end the second capture group

/

We're finally done with the mattern to match!

Now we'll start the replacement pattern.

\1

Insert the full content of the first capture group

(this is also called a "backreference" if you want

to google for a more detailed explanation).

So insert the whole first command up to the newline

after it.

[no results]

After the newline, insert your desired string.

\r

insert a carriage return here (I thought this should be

\n for a newline, but that made vim insert a null instead)

\2

insert the second capture group (that's just the second prompt)

/

end of the substitute pattern

Of course, if you have a different prompt, substitute it for "% ". If you have a complicated prompt that includes time of day or something, you'll have to use a slightly more complicated match pattern to match it.

Tags: regexp, shell, CLI, linux, editors
[ 14:34 Oct 12, 2008    More linux/editors | permalink to this entry | ]

Smart Wrapping with Greedy and Non-Greedy Regular Expressions

I had a chance to spend a day at the AGU conference last week. The American Geophysical Union is a fabulous conference -- something like 14,000 different talks over the course of the week, on anything related to earth or planetary sciences -- geology, solar system astronomy, atmospheric science, geophysics, geochemistry, you name it.

I have no idea how regular attendees manage the information overload of deciding which talks to attend. I wasn't sure how I would, either, but I started by going through the schedule for the day I'd be there, picking out a (way too long) list of potentially interesting talks, and saving them as lines in a file.

Now I had a file full of lines like:

1020      U22A    MS 303  Terrestrial Impact Cratering: New Insights Into the Cratering Process From Geophysics and Geochemistry II

Fine, except that I couldn't print out something like that -- printers stop at 80 columns. I could pass it through a program like "fold" to wrap the long lines, but then it would be hard to scan through quickly to find the talk titles and room numbers. What I really wanted was to wrap it so that the above line turned into something like:

1020      U22A    MS 303  Terrestrial Impact Cratering: New Insights
                          Into the Cratering Process From Geophysics
                          and Geochemistry II

But how to do that? I stared at it for a while, trying to figure out whether there was a clever vim substitute that could handle it. I asked on a couple of IRC channels, just in case there was some amazing Linux smart-wrap utility I'd never heard of. I was on the verge of concluding that the answer was no, and that I'd have to write a python script to do the wrapping I wanted, when Mikael emitted a burst of line noise:
%s/\(.\{72\}\)\(.*\)/\1^M^I^I^I\2/

Only it wasn't line noise. Seems Mikael just happened to have been reading about some of the finer points of vim regular expressions earlier that day, and he knew exactly the trick I needed -- that .\{72\}, which matches lines that are at least 72 characters long. And amazingly, that expression did something very close to what I wanted.

Or at least the first step of it. It inserts the first line break, turning my line into

1020      U22A    MS 303  Terrestrial Impact Cratering: New Insights
                          Into the Cratering Process From Geophysics and Geochemistry II

but I still needed to wrap the second and subsequent lines.

But that was an easier problem -- just do essentially the same thing again, but limit it to only lines starting with a tab. After some tweaking, I arrived at exactly what I wanted:

%s/^\(.\{,65\}\) \(.*\)/\1^M^I^I^I\2/

%g/^^I^I^I.\{58\}/s/^\(.\{,55\}\) \(.*\)/\1^M^I^I^I\2/

I had to run the second line two or three times to wrap the very long lines.

Devdas helpfully translated the second one into English: "You have 3 tabs, followed by 58 characters, out of which you match the first 55 and put that bit in $1, and the capture the remaining in $2, and rewrite to $1 newline tab tab tab $2."

Here's a more detailed breakdown:

Line one:

%	Do this over the whole file
s/	Begin global substitute
^	Start at the beginning of the line
\(	Remember the result of the next match
.\{,65\}_	Look for up to 65 characters with a space at the end
\) \(	End of remembered pattern #1, skip a space, and start remembered pattern #2
.*\)	Pattern #2 includes everything to the end of the line
/	End of matched pattern; begin replacement pattern
\1^M	Insert saved pattern #1 (the first 65 lines ending with a space) followed by a newline
^I^I^I\2	On the second line, insert three tabs then saved pattern #2
/	End replacement pattern

Line two:

%g/	Over the whole file, only operate on lines with this pattern
^^I^I^I	Lines starting with three tabs
.\{58\}/	After the tabs, only match lines that still have at least 58 characters (this guards against wrapping already wrapped lines when it's run repeatedly)
s/	Begin global substitute
^	Start at the beginning of the line
\(	Remember the result of the next match
.\{,55\}	Up to 55 characters
\) \(	End of remembered pattern #1, skip a space, and start remembered pattern #2
.*\)	Pattern #2 includes everything to the end of the line
/	End of matched pattern; begin replacement pattern
\1^M	The first pattern (up to 55 chars) is one line
^I^I^I\2	Three tabs then the second pattern
/	End replacement pattern

Greedy and non-greedy brace matches

The real key is those curly-brace expressions, \{,65\} and \{58\} -- that's how you control how many characters vim will match and whether or not the match is "greedy". Here's how they work (thanks to Mikael for explaining).

The basic expression is {M,N} -- it means between M and N matches of whatever precedes it. (Vim requires that the first brace be escaped -- \{}. Escaping the second brace is optional.) So .{M,N} can match anything between M and N characters but "prefer" N, i.e. try to match as many as possible up to N. To make it "non-greedy" (match as few as possible, "preferring" M), use .{-M,N}

You can leave out M, N, or both; M defaults to 0 while N defaults to infinity. So {} is short for {0,∞} and is equivalent to *, while {-} means {-0,∞}, like a non-greedy version of *.

Given the string: one, two, three, four, five

,.\{},	matches , two, three, four,
,.\{-},	matches , two,
,.\{5,},	matches , two, three, four,
,.\{-5,},	matches , two, three,
,.\{,2},	matches nothing
,.\{,7},	matches , two,
,.\{5,7},	matches , three,

Of course, this syntax is purely for vim; regular expressions are unfortunately different in sed, perl and every other program. Here's a fun table of regexp terms in various programs.

Tags: linux, editors, regexp
[ 12:44 Dec 20, 2007    More linux/editors | permalink to this entry | ]

Emacs with Long Lines

I don't like composing text documents in word processors like Open Office. Call it a quirk if you like, but I find them intrusive: they take up a lot of CPU and memory, they take up a lot of window space for stuff I don't need while I'm writing (all those margins and rulers and toolbars and such) making it hard to compare two documents at once, and they tend to have intrusive focus behavior (like popping windows to the front when I didn't ask for it).

So when I need to write a paper (or a book), I prefer to compose in a text editor like vim or emacs, something that won't get in the way of my train of thought. When it's mostly written and ready to format, then I start up the big heavyweight word processor and import or paste the text into it.

(For those of you who think I'm insane and should just live in Open Office all day, the same problem comes up for people who do a lot of composing for web applications, such as an online blog, gmail, a web forum, or a wiki, and for people who want a choice of editor for their GUI mail app.)

Fine, but that introduces a problem. See, text editors have a fixed line width (typically 80 characters, though of course you can adjust this) and paragraphs are usually separated by blank lines (two newline characters together). Word processors expect each paragraph to be one long line for the whole paragraph, and line breaks are used as paragraph breaks (but you only want one of them, not two). How do you reconcile these two models in order to paste plaintext from an editor into a word processor?

Several years ago when I first encountered this problem, I investigated solutions in both vim and emacs (oddly enough, I'm an editor agnostic and equally happy in either one).

For vim, I never did find a solution to the problem, so that settled the editor choice for me. Perhaps some vim expert can let me know what I missed.

For emacs, I found longlines-mode, a hack which lets long lines appear to be wrapped while you're editing them even though they're really not. Apparently Wikipedia has this issue and some Wikipedia contributors use longlines-mode too. (That page also has brief notes on alternate solutions.)

I used longlines-mode for a long time, and it's more or less functional, but I was never really happy with it. It turns out to have some pretty annoying bugs which I was forever needing to work around, and it doesn't solve the blank-lines problem -- you still need to delete blank lines before or after pasting.

Yesterday I was working on an essay for a class I'm taking and decided I'd had enough of longlines-mode and wanted a better solution. I poked around and chatted with the nice folks on #emacs (hoping that someone had come up with a better solution, but no one knew of one) and based on some ideas they had, I came up with one of my own.

My new method is to edit the text file normally: line breaks where they look good, blank lines to separate paragraphs. When I'm finished writing and ready to paste, I run M-x wp-munge, which calls up a very simple function I wrote and added to my .emacs:

;; For composing in emacs then pasting into a word processor,
;; this un-fills all the paragraphs (i.e. turns each paragraph
;; into one very long line) and removes any blank lines that
;; previously separated paragraphs.
;;
(defun wp-munge () "un-fill paragraphs and remove blank lines" (interactive)
  (let ((save-fill-column fill-column))
    (set-fill-column 1000000)
    (mark-whole-buffer)
    (fill-individual-paragraphs (point-min) (point-max))
    (delete-matching-lines "^$")
    (set-fill-column save-fill-column) ))

So simple! Why didn't I think of doing it that way before?

Tags: emacs, editors
[ 21:10 Feb 19, 2007    More linux/editors | permalink to this entry | ]

Linkifying with Regular Expressions

I had a page of plaintext which included some URLs in it, like this:

Tour of the Hayward Fault
http://www.mcs.csuhayward.edu/~shirschf/tour-1.html

Technical Reports on Hayward Fault
http://quake.usgs.gov/research/geology/docs/lienkaemper_docs06.htm

I wanted to add links around each of the urls, so that I could make it part of a web page, more like this:

Tour of the Hayward Fault
http://www.mcs.csu hayward.edu/~shirschf/tour-1.html

Technical Reports on Hayward Fault
htt p://quake.usgs.gov/research/geology/docs/lienkaemper_docs06.htm

Surely there must be a program to do this, I thought. But I couldn't find one that was part of a standard Linux distribution.

But you can do a fair job of linkifying just using a regular expression in an editor like vim or emacs, or by using sed or perl from the commandline. You just need to specify the input pattern you want to change, then how you want to change it.

Here's a recipe for linkifying with regular expressions.

Within vim:

:%s_\(https\=\|ftp\)://\S\+_<a href="&">&</a>_

If you're new to regular expressions, it might be helpful to see a detailed breakdown of why this works:

:

Tell vim you're about to type a command.

%

The following command should be applied everywhere in the file.

s_

Do a global substitute, and everything up to the next underscore will represent the pattern to match.

\(

This will be a list of several alternate patterns.

http

If you see an "http", that counts as a match.

s\=

Zero or one esses after the http will match: so http and https are okay, but httpsssss isn't.

\|

Here comes another alternate pattern that you might see instead of http or https.

ftp

URLs starting with ftp are okay too.

\)

We're done with the list of alternate patterns.

://

After the http, https or ftp there should always be a colon-slash-slash.

\S

After the ://, there must be a character which is not whitespace.

\+

There can be any number of these non-whitespace characters as long as there's at least one. Keep matching until you see a space.

_

Finally, the underscore that says this is the end of the pattern to match. Next (until the final underscore) will be the expression which will replace the pattern.

<a href="&">

An ampersand, &, in a substitute expression means "insert everything that was in the original pattern". So the whole url will be inserted between the quotation marks.

&</a>

Now, outside the <a href="..."> tag, insert the matched url again, and follow it with a </a> to close the tag.

_

The final underscore which says "this is the end of the replacement pattern". We're done!

Linkifying from the commandline using sed

Sed is a bit trickier: it doesn't understand \S for non-whitespace, nor = for "zero or one occurrence". But this expression does the trick:
sed -e 's_\(http\|https\|ftp\)://[^ \t]\+_<a href="&">&</a>_' <infile.txt >outfile.html

Addendum: George Riley tells me about VST for Vim 7, which looks like a nice package to linkify, htmlify, and various other useful things such as creating HTML presentations. I don't have Vim 7 yet, but once I do I'll definitely check out VST.

Tags: linux, editors, pipelines, regexp, shell, CLI
[ 13:40 May 14, 2006    More linux/editors | permalink to this entry | ]

Emacs: Typing dashes in html mode

What to do with a few extra hours in a boring motel with no net access? How about digging into fixing one of Emacs' more annoying misfeatures?

Whenever I edit an html file using emacs, I find I have to stay away from double dashes -- I can't add a phrase such as this one. If I forget and type a phrase with a double dash, then as soon as I get to the end of that line and emacs decides it's time to wrap to the next line, it "helpfully" treats the double dashes as a comment, and indents the next line to the level where the dashes were, adding another set of dashes. I've googled, I've asked on emacs IRC help channels, but there doesn't seem to be any way out. (I guess no one else ever uses double dashes in html files?)

It's frustrating: I like using double dashes now and then. And aside from the occasional boneheaded misfeature like this one, I like using emacs. But the dash problem been driving me nuts for a long time now. So I finally dug into the code to cure it.

First, the file is sgml-mode.el, so don't bother searching anything with html in the name. On my system it's /usr/share/emacs/21.4/lisp/textmodes/sgml-model.el. Edit that file and search for "--" and the first thing you'll find (well, after the file's preamble comments) is a comment in the definition of "sgml-specials" saying that if you include ?- in the list of specials, it will hork the typing of double dashes, so that's normally left out.

A clue! Perhaps some Debian or Ubuntu site file has changed sgml-specials for me, and all I need to do is change it back! So I typed

M-x describe-variable sgml-specials

to see the current setting.

Um ... it's set to "34". That's not very helpful. I haven't a clue how that translates to the list of characters I see in sgml-mode.el. Forget about that approach for now.

Searching through the file for the string "comment" got me a few more hits, and I tried commenting out various comment handling lines until the evil behavior went away. (I had to remove sgml-mode.elc first, otherwise emacs wouldn't see any changes I made to sgml-mode.el. If you haven't done much elisp hacking, the .el is the lisp source, while the .elc is a byte-compiled version which loads quicker but isn't intended to be edited by humans. For Java programmers, the .elc is sort of like a .class file.)

Commenting out these four lines did the trick:

  (set (make-local-variable 'font-lock-syntactic-keywords)
       '(("\\(<\\)! *--.*-- *\\(>\\)" (1 "!") (2 "!"))))
  ;; This will allow existing comments within declarations to be
  ;; recognized.
  (set (make-local-variable 'comment-start-skip) "\\(?:\\)?")

To regenerate the .elc file so sgml-mode will load faster, I ran emacs as root from the directory sgml-mode.el was in, and typed:

M-x byte-compile-file sgml-mode.el

All better! And now I know where to find documentation for all those useful-looking, but seemingly undocumented, keyboard shortcuts that go along with emacs' html mode. Just search in the file for html-mode-map, and you'll find all sorts of useful stuff.

For instance, that typing Ctrl-C Ctrl-C followed by various letters: u gets you an unordered list, h gets you an href tag, i an image tag, and so on, with the cursor positioned where you want to type next.

It doesn't seem to offer any basic inline formatting (like <i> or <em>), alas; but of course that's easy to add by editing the file (or maybe even in .emacs). To add an <em> tag, add this line to html-mode-map:

    (define-key map "\C-c\C-ce" 'html-em)

then add this function somewhere near where html-headline-1 and friends are defined:

(define-skeleton html-em
  "HTML emphasis tags."
  nil
  "" _ "")

Of course, you can define any set of tags you use often, not just <em>.

HTML mode in emacs should be much more fun and less painful now!

Update: If you don't want to modify the files as root, it also works fine to copy sgml-mode.el to wherever you keep personal elisp files. For instance, put them in a directory called ~/.emacs-lisp then add this to your .emacs:
(setq load-path (cons "~/.emacs-lisp/" load-path))

Tags: emacs, editors
[ 22:48 Mar 29, 2006    More linux/editors | permalink to this entry | ]

Helpful Vim Tip: Finding Syntax for Colors

An upgrade from woody to sarge introduced a new problem with editing mail messages in vim: Subject lines appeared in yellow, against my light grey background, so they weren't readable any more.

Vim color files have always been a mystery to me. I have one which I adapted from one of the standard color schemes, but I've never been clear what the legal identifiers are or how to find out. But I changed both places where it said "ctermfg=Yellow" to another color, and nothing changed, so this time I had to find out.

Fortunately a nice person on #vim suggested :he synID (he is short for "help", of course) which told me all I needed to know. Put the cursor on the errant line and type: :echo synIDattr(synID(line("."), col("."), 1), "name")

That told me that the Subject line was syntax class "mailSubject". So I tried (copying other lines in my color file) adding this line:

hi mailSubject term=underline ctermfg=Red guifg=Red

and now all is happy again in vim land. I wish I'd learned that synID trick a long time ago!

Tags: vim, color, editors, tips
[ 10:59 Jun 22, 2005    More linux/editors | permalink to this entry | ]

Tweaking Emacs' Text Indent: Don't Indent So Aggressively

Encouraged by my success a few days ago at finally learning how to disable vim's ctrl-spacebar behavior, the next day I went back to an emacs problem that's been bugging me for a while: in text mode, newline-and-indent always wants to indent the first line of a text file (something I almost never want), and skips blank lines when calculating indent (so starting a new paragraph doesn't reset the indent back to zero).

I had already googled to no avail, and had concluded that the only way was to write a new text-indent function which could be bound to the return key in the text mode hook.

This went fairly smoothly: I got a little help in #emacs with checking the pattern immediately before the cursor (though I turned out not to need that after all) and for the function called "bobp" (beginning of buffer predicate). Here's what I ended up with:

(defun newline-and-text-indent ()
  "Insert a newline, then indent the next line sensibly for text"
  (interactive)
  (if (or (bobp)
          (looking-at "^$"))
      (newline)
      (newline-and-indent)
  ))
(defun text-indent-hook ()
  (local-set-key "\C-m" 'newline-and-text-indent)
  )
(setq text-mode-hook 'text-indent-hook)

It seems to work fine. For the curious, here's my current .emacs

Tags: emacs, editors
[ 14:03 Feb 19, 2005    More linux/editors | permalink to this entry | ]

Turning off Ctrl-Space in Vim

One of those niggling problems that has plagued me for a long time: in the editor vim, if I'm typing along in insert mode and instead of a space I accidentally hit control-space, vim inserts a bunch of text I didn't want, then exits insert mode. Meanwhile I'm still merrily typing away, typing what are now vim comments which invariably end up deleting the last two paragraphs I typed then doing several more operations which end up erasing the undo buffer so I can't get those paragraphs back.

Ctrl-space inserts a null character (you can verify this by putting it in a file and running od -xc on it). I've done lots of googling in the past, but it's hard to google on strings like " " or even "space" or "null", and nobody I asked had a clue what this function was called (it turns out it re-inserts whatever the last inserted characters were) so I couldn't google on the function name.

Vim's help suggests that <Nul>, <Char-0>, or <C-^V > should do it. I tried them with map, vmap, cmap, and nmap, to no avail. I also tried <C-@> since that's a historical way of referring to the null character, googling found some references to that in vim, and that's how it displays if I type it in vim.

I finally found #vim on freenode, and asked there. Last night nobody knew, but this morning, p0g found the problem: I needed to use imap, not the map/vmap/cmap/nmap I'd been using.

So here, perserved for google posterity in case other people are plagued by this problem, is the answer:

imap <Nul> <Space>

For good measure, I also mapped the character to no-op in all the other modes as well:

map  <Nul> <Nop>
vmap <Nul> <Nop>
cmap <Nul> <Nop>
nmap <Nul> <Nop>

My current .vimrc.

Tags: vim, editors, tips
[ 11:24 Feb 17, 2005    More linux/editors | permalink to this entry | ]

Emacs Color Themes

A nifty emacs trick I learned about today: ColorThemes.

Instead of the old hacked-together color collection I've been using in emacs, I can load color-theme.el and choose from lots of different color schemes.

I added these lines to .emacs:

(require 'font-lock)
(if (fboundp 'global-font-lock-mode) (global-font-lock-mode 1))
(load "~/.emacs-lisp/color-theme.el")
(color-theme-ramangalahy)  ;; pick a favorite theme

The disadvantage is that color-theme.el is fifteen thousand lines long! So I'll probably make a local version that strips out all but the theme I actually use (then I can customize that).

The (global-font-lock-mode 1) tells emacs to use syntax highlighting on every file, not just certain types. So now I get at least some highlighting even in html files, though it still doesn't seem to be able to highlight like vim does (e.g. different colors for text inside <b> or <b> tags).

Tags: emacs, editors
[ 18:57 Feb 03, 2005    More linux/editors | permalink to this entry | ]

Nifty emacs hack: how to wrap only certain files

For a long time I've wanted some, but not all, text and html files to line-wrap automatically in emacs. For instance, it drives me nuts when I edit a system configuration file and it wraps each long line, or when I edit an html file containing lots of long links and it keeps wrapping between the <a and the href=. But for files which are mostly text (such as these blog entries), I want line wrapping.

I'd been trying to do this with html-mode-hook and text-mode-hook, then checking the filename and calling (auto-fill-mode) if appropriate, but it wasn't working, because buffer-file-name isn't always defined at the time the mode hook is called. (No one seems to know why.) The buffer name seems to be defined at that point, but it doesn't contain path information so I can't say "Use wrapping for anything under ~/Docs" or "Don't wrap anything in /etc".

But with some help from sachac and the nice folks on #emacs I came up with a much better solution, and it's way simpler than the mode-hook approach: derived modes.

I set up two new modes, called html-wrap-mode and text-wrap-mode, which are the same as html-mode and text-mode except that they turn on auto-fill. Then I use the easy auto-mode-alist mechanism, which already does string matching on the filename, to call these modes, instead of the regular text and html modes, based on the extension or some other aspect of the file's pathname. Here's what I added to .emacs:

;; Want auto-fill-mode for some text and html files, but not all.
;; So define two derived modes for that, and we'll use auto-mode-alist
;; to choose them based on filename.
(define-derived-mode html-wrap-mode html-mode "HTML wrap mode"
  (auto-fill-mode))
(define-derived-mode text-wrap-mode text-mode "Text wrap mode"
  (auto-fill-mode))

(setq auto-mode-alist
      (cons '("\\.blx$" . html-wrap-mode)
      (cons '("Docs/.*.html$" . html-wrap-mode)
      (cons '("Docs/" . text-wrap-mode)
            auto-mode-alist) ) ) )

Here's my current .emacs.

I wonder if vim has a way to do this?

Tags: editors, emacs
[ 23:30 Jan 13, 2005    More linux/editors | permalink to this entry | ]