Shallow Thoughts : : editors

Akkana's Musings on Open Source Computing and Technology, Science, and Nature.

Sat, 18 Feb 2017

Highlight and remove extraneous whitespace in emacs

I recently got annoyed with all the trailing whitespace I saw in files edited by Windows and Mac users, and in code snippets pasted from sites like StackOverflow. I already had my emacs set up to indent with only spaces:

(setq-default indent-tabs-mode nil)
(setq tabify nil)
and I knew about M-x delete-trailing-whitespace ... but after seeing someone else who had an editor set up to show trailing spaces, and tabs that ought to be spaces, I wanted that too.

To show trailing spaces is easy, but it took me some digging to find a way to control the color emacs used:

;; Highlight trailing whitespace.
(setq-default show-trailing-whitespace t)
(set-face-background 'trailing-whitespace "yellow")

I also wanted to show tabs, since code indented with a mixture of tabs and spaces, especially if it's Python, can cause problems. That was a little harder, but I eventually found it on the EmacsWiki: Show whitespace:

;; Also show tabs.
(defface extra-whitespace-face
  '((t (:background "pale green")))
  "Color for tabs and such.")

(defvar bad-whitespace
  '(("\t" . 'extra-whitespace-face)))

While I was figuring this out, I got some useful advice related to emacs faces on the #emacs IRC channel: if you want to know why something is displayed in a particular color, put the cursor on it and type C-u C-x = (the command what-cursor-position with a prefix argument), which displays lots of information about whatever's under the cursor, including its current face.

Once I had my colors set up, I found that a surprising number of files I'd edited with vim had trailing whitespace. I would have expected vim to be better behaved than that! But it turns out that to eliminate trailing whitespace, you have to program it yourself. For instance, here are some recipes to Remove unwanted spaces automatically with vim.

Tags: ,
[ 16:41 Feb 18, 2017    More linux/editors | permalink to this entry | comments ]

Mon, 13 Feb 2017

Emacs: Initializing code files with a template

Part of being a programmer is having an urge to automate repetitive tasks.

Every new HTML file I create should include some boilerplate HTML, like <html><head></head></body></body></html>. Every new Python file I create should start with #!/usr/bin/env python, and most of them should end with an if __name__ == "__main__": clause. I get tired of typing all that, especially the dunderscores and slash-greater-thans.

Long ago, I wrote an emacs function called newhtml to insert the boilerplate code:

(defun newhtml ()
  "Insert a template for an empty HTML page"
  (interactive)
  (insert "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\">\n"
          "<html>\n"
          "<head>\n"
          "<title></title>\n"
          "</head>\n\n"
          "<body>\n\n"
          "<h1></h1>\n\n"
          "<p>\n\n"
          "</body>\n"
          "</html>\n")
  (forward-line -11)
  (forward-char 7)
  )

The motion commands at the end move the cursor back to point in between the <title> and </title>, so I'm ready to type the page title. (I should probably have it prompt me, so it can insert the same string in title and h1, which is almost always what I want.)

That has worked for quite a while. But when I decided it was time to write the same function for python:

(defun newpython ()
  "Insert a template for an empty Python script"
  (interactive)
  (insert "#!/usr/bin/env python\n"
          "\n"
          "\n"
          "\n"
          "if __name__ == '__main__':\n"
          "\n"
          )
  (forward-line -4)
  )
... I realized that I wanted to be even more lazy than that. Emacs knows what sort of file it's editing -- it switches to html-mode or python-mode as appropriate. Why not have it insert the template automatically?

My first thought was to have emacs run the function upon loading a file. There's a function with-eval-after-load which supposedly can act based on file suffix, so something like (with-eval-after-load ".py" (newpython)) is documented to work. But I found that it was never called, and couldn't find an example that actually worked.

But then I realized that I have mode hooks for all the programming modes anyway, to set up things like indentation preferences. Inserting some text at the end of the mode hook seems perfectly simple:

(add-hook 'python-mode-hook
          (lambda ()
            (electric-indent-local-mode -1)
            (font-lock-add-keywords nil bad-whitespace)
            (if (= (buffer-size) 0)
                (newpython))
            (message "python hook")
            ))

The (= (buffer-size) 0) test ensures this only happens if I open a new file. Obviously I don't want to be auto-inserting code inside existing programs!

HTML mode was a little more complicated. I edit some files, like blog posts, that use HTML formatting, and hence need html-mode, but they aren't standalone HTML files that need the usual HTML template inserted. For blog posts, I use a different file extension, so I can use the elisp string-suffix-p to test for that:

  ;; s-suffix? is like Python endswith
  (if (and (= (buffer-size) 0)
           (string-suffix-p ".html" (buffer-file-name)))
      (newhtml) )

I may eventually find other files that don't need the template; if I need to, it's easy to add other tests, like the directory where the new file will live.

A nice timesaver: open a new file and have a template automatically inserted.

Tags: , ,
[ 09:52 Feb 13, 2017    More linux/editors | permalink to this entry | comments ]

Thu, 09 Jun 2016

Visual diffs and file merges with vimdiff

I needed to merge some changes from a development file into the file on the real website, and discovered that the program I most often use for that, meld, is in one of its all too frequent periods where its developers break it in ways that make it unusable for a few months. (Some of this is related to GTK, which is a whole separate rant.)

That led me to explore some other diff/merge alternatives. I've used tkdiff quite a bit for viewing diffs, but when I tried to use it to merge one file into another I found its merge just too hard to use. Likewise for emacs: it's a wonderful editor but I never did figure out how to get ediff to show diffs reliably, let alone merge from one file to another.

But vimdiff looked a lot easier and had a lot more documentation available, and actually works pretty well.

I normally run vim in an xterm window, but for a diff/merge tool, I want a very wide window which will show the diffs side by side. So I used gvimdiff instead of regular vimdiff: gvimdiff docs.dev/filename docs.production/filename

Configuring gvimdiff to see diffs

gvimdiff initially pops up a tiny little window, and it ignores Xdefaults. Of course you can resize it, but who wants to do that every time? You can control the initial size by setting the lines and columns variables in .vimrc. About 180 columns by 60 lines worked pretty well for my fonts on my monitor, showing two 80-column files side by side. But clearly I don't want to set that in .vimrc so that it runs every time I run vim; I only want that super-wide size when I'm running a side-by-side diff.

You can control that by checking the &diff variable in .vimrc:

if &diff
    set lines=58
    set columns=180
endif

If you do decide to resize the window, you'll notice that the separator between the two files doesn't stay in the center: it gives you lots of space for the right file and hardly any for the left. Inside that same &diff clause, this somewhat arcane incantation tells vim to keep the separator centered:

    autocmd VimResized * exec "normal \<C-w>="

I also found that the colors, in the vim scheme I was using, made it impossible to see highlighted text. You can go in and edit the color scheme and make your own, of course, but an easy way quick fix is to set all highlighting to one color, like yellow, inside the if $diff section:

    highlight DiffAdd    cterm=bold gui=none guibg=Yellow
    highlight DiffDelete cterm=bold gui=none guibg=Yellow
    highlight DiffChange cterm=bold gui=none guibg=Yellow
    highlight DiffText   cterm=bold gui=none guibg=Yellow

Merging changes

Okay, once you can view the differences between the two files, how do you merge from one to the other? Most online sources are quite vague on that, but it's actually fairly easy:
]c jumps to the next difference
[c jumps to the previous difference
dp makes them both look like the left side (apparently stands for diff put
do makes them both look like the right side (apparently stands for diff obtain

The only difficult part is that it's not really undoable. u (the normal vim undo keystroke) works inconsistently after dp: the focus is generally in the left window, so u applies to that window, while dp modified the right window and the undo doesn't apply there. If you put this in your .vimrc

nmap du :wincmd w<cr>:normal u<cr>:wincmd w<cr>
then you can use du to undo changes in the right window, while u still undoes in the left window. So you still have to keep track of which direction your changes are going.

Worse, neither undo nor this du command restores the highlighting showing there's a difference between the two files. So, really, undoing should be reserved for emergencies; if you try to rely on it much you'll end up being unsure what has and hasn't changed.

In the end, vimdiff probably works best for straightforward diffs, and it's probably best get in the habit of always merging from right to left, using do. In other words, run vimdiff file-to-merge-to file-to-merge-from, and think about each change before doing it to make it less likely that you'll need to undo.

And hope that whatever silly transient bug in meld drove you to use vimdiff gets fixed quickly.

Tags: , ,
[ 20:10 Jun 09, 2016    More linux/editors | permalink to this entry | comments ]

Sat, 12 Dec 2015

Emacs rich-text mode: coloring and styling plain text

I use emacs a lot for taking notes, during meetings, while watching lectures in a MOOC, or while researching something.

But one place where emacs falls short is highlighting. For instance, if I paste a section of something I'm researching, then I want to add a comment about it, to differentiate the pasted part from my added comments, I have to resort to horrible hacks like "*********** My comment:". It's like the stuff Outlook users put in emails because they can't figure out how to quote.

What I really want is a simple rich-text mode, where I can highlight sections of text by changing color or making it italic, bold, underlined.

Enter enriched-mode. Start it with M-x enriched-mode and then you can apply some styles with commands like M-o i for italic, M-o b for bold, etc. These styles may or may not be visible depending on the font you're using; for instance, my font is already bold and emacs isn't smart enough to make it bolder, the way some programs are. So if one style doesn't work, try another one.

Enriched mode will save these styles when you save the file, with a markup syntax like <italic>This text is in italic.</italic> When you load the file, you'll just see the styles, not the markup.

Colors

But they're all pretty subtle. I still wanted colors, and none of the documentation tells you much about how to set them.

I found a few pages saying that you can change the color of text in an emacs buffer using the Edit menu, but I hide emacs's menus since I generally have no use for them: emacs can do everything from the keyboard, one of the things I like most about it, so why waste space on a menu I never use? I do that like this:

(tool-bar-mode 0)
(menu-bar-mode 0)

It turns out that although the right mouse button just extends the selection, Control-middleclick gives a context menu. Whew! Finally a way to change colors! But it's not at all easy to use: Control-middleclick, mouse over Foreground Color, slide right to Other..., click, and the menu goes away and now there's a prompt in the minibuffer where you can type in a color name.

Colors are saved in the file with a syntax like: <x-color><param>red</param>This text is in red.</x-color>

All that clicking is a lot of steps, and requires taking my hands off the keyboard. How do I change colors in an easier, keyboard driven way? I drew a complete blank with my web searches. A somewhat irritable person on #emacs eventually hinted that I should be using overlays, and I eventually figured out how to set overlay colors ((overlay-put (make-overlay ...)) turned out to be the way to do that) but it was a complete red herring: enriched-mode doesn't pay any attention to overlay colors. I don't know what overlays are useful for, but it's not that.

But in emacs, you can find out what's bound to a key with describe-key. Maybe that works for mouse clicks too? I ran describe-key, held down Control, clicked the middle button -- the context menu came up -- then navigated to Foreground Color and Other... and discovered that it's calling (facemenu-set-foreground COLOR &optional START END).

Binding to keys

Finally, a function I can bind to a key! COLOR is just a string, like "red". The documentation implies that START and END are optional, and that the function will apply to the selected region if there is one. But in practice, if you don't specify START and END, nothing happens, so you have to specify them. (region-beginning) and (region-end) work if you have a selected region.

Similarly, I learned that Face->italic from that same menu calls (facemenu-set-italic), and likewise for bold, underline etc. They work on the selected region.

But what if there's no region defined? I decided it might be nice to be able to set styles for the current line, without selecting it first. I can use (line-beginning-position) and (line-end-position) for START and END. So I wrote a wrapper function. For that, I didn't want to use specific functions like (facemenu-set-italic); I wanted to be able pass a property like "italic" to my wrapper function.

I found a way to do that: (put-text-property START END 'italic). But that wasn't quite enough, because put-text-property replaces all properties; you can't make something both italic and bold. To add a property without removing existing ones, use (add-text-properties START END (list 'face 'italic)).

So here's the final code that I put in my .emacs. I was out of excuses to procrastinate, and my enriched-mode bindings worked fine for taking notes on the project which had led to all this procrastination.

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Text colors/styles. You can use this in conjunction with enriched-mode.
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

;; rich-style will affect the style of either the selected region,
;; or the current line if no region is selected.
;; style may be an atom indicating a rich-style face,
;; e.g. 'italic or 'bold, using
;;   (put-text-property START END PROPERTY VALUE &optional OBJECT)
;; or a color string, e.g. "red", using
;;   (facemenu-set-foreground COLOR &optional START END)
;; or nil, in which case style will be removed.
(defun rich-style (style)
  (let* ((start (if (use-region-p)
                    (region-beginning) (line-beginning-position)))
                    
         (end   (if (use-region-p)
                    (region-end)  (line-end-position))))
    (cond
     ((null style)      (set-text-properties start end nil))
     ((stringp style)   (facemenu-set-foreground style start end))
     (t                 (add-text-properties start end (list 'face style)))
     )))

(defun enriched-mode-keys ()
  (define-key enriched-mode-map "\C-ci"
    (lambda () (interactive)    (rich-style 'italic)))
  (define-key enriched-mode-map "\C-cB"
    (lambda () (interactive)    (rich-style 'bold)))
  (define-key enriched-mode-map "\C-cu"
    (lambda () (interactive)    (rich-style 'underline)))
  (define-key enriched-mode-map "\C-cr"
    (lambda () (interactive)    (rich-style "red")))
  ;; Repeat for any other colors you want from rgb.txt

  (define-key enriched-mode-map (kbd "C-c ")
    (lambda () (interactive)    (rich-style nil)))
  )
(add-hook 'enriched-mode-hook 'enriched-mode-keys)

Tags: ,
[ 14:48 Dec 12, 2015    More linux/editors | permalink to this entry | comments ]

Sun, 14 Sep 2014

Global key bindings in Emacs

Global key bindings in emacs. What's hard about that, right? Just something simple like

(global-set-key "\C-m" 'newline-and-indent)
and you're all set.

Well, no. global-set-key gives you a nice key binding that works ... until the next time you load a mode that wants to redefine that key binding out from under you.

For many years I've had a huge collection of mode hooks that run when specific modes load. For instance, python-mode defines \C-c\C-r, my binding that normally runs revert-buffer, to do something called run-python. I never need to run python inside emacs -- I do that in a shell window. But I fairly frequently want to revert a python file back to the last version I saved. So I had a hook that ran whenever python-mode loaded to override that key binding and set it back to what I'd already set it to:

(defun reset-revert-buffer ()
  (define-key python-mode-map "\C-c\C-r" 'revert-buffer) )
(setq python-mode-hook 'reset-revert-buffer)

That worked fine -- but you have to do it for every mode that overrides key bindings and every binding that gets overridden. It's a constant chase, where you keep needing to stop editing whatever you wanted to edit and go add yet another mode-hook to .emacs after chasing down which mode is causing the problem. There must be a better solution.

A web search quickly led me to the StackOverflow discussion Globally override key bindings. I tried the techniques there; but they didn't work.

It took a lot of help from the kind folks on #emacs, but after an hour or so they finally found the key: emulation-mode-map-alists. It's only barely documented -- the key there is "The “active” keymaps in each alist are used before minor-mode-map-alist and minor-mode-overriding-map-alist" -- and there seem to be no examples anywhere on the web for how to use it. It's a list of alists mapping names to keymaps. Oh, clears it right up! Right?

Okay, here's what it means. First you define a new keymap and add your bindings to it:

(defvar global-keys-minor-mode-map (make-sparse-keymap)
  "global-keys-minor-mode keymap.")

(define-key global-keys-minor-mode-map "\C-c\C-r" 'revert-buffer)
(define-key global-keys-minor-mode-map (kbd "C-;") 'insert-date)

Now define a minor mode that will use that keymap. You'll use that minor mode for basically everything.

(define-minor-mode global-keys-minor-mode
  "A minor mode so that global key settings override annoying major modes."
  t "global-keys" 'global-keys-minor-mode-map)

(global-keys-minor-mode 1)

Now build an alist consisting of a list containing a single dotted pair: the name of the minor mode and the keymap.

;; A keymap that's supposed to be consulted before the first
;; minor-mode-map-alist.
(defconst global-minor-mode-alist (list (cons 'global-keys-minor-mode
                                              global-keys-minor-mode-map)))

Finally, set emulation-mode-map-alists to a list containing only the global-minor-mode-alist.

(setf emulation-mode-map-alists '(global-minor-mode-alist))

There's one final step. Even though you want these bindings to be global and work everywhere, there is one place where you might not want them: the minibuffer. To be honest, I'm not sure if this part is necessary, but it sounds like a good idea so I've kept it.

(defun my-minibuffer-setup-hook ()
  (global-keys-minor-mode 0))
(add-hook 'minibuffer-setup-hook 'my-minibuffer-setup-hook)

Whew! It's a lot of work, but it'll let me clean up my .emacs file and save me from endlessly adding new mode-hooks.

Tags: , ,
[ 16:46 Sep 14, 2014    More linux/editors | permalink to this entry | comments ]

Sun, 15 Jun 2014

Vim: Set wrapping and indentation according to file type

Although I use emacs for most of my coding, I use vim quite a lot too, for quick edits, mail messages, and anything I need to edit when logged onto a remote server. In particular, that means editing my procmail spam filter files on the mail server.

The spam rules are mostly lists of regular expression patterns, and they can include long lines, such as:
gift ?card .*(Visa|Walgreen|Applebee|Costco|Starbucks|Whitestrips|free|Wal.?mart|Arby)

My default vim settings for editing text, including line wrap, don't work if get a flood of messages offering McDonald's gift cards and decide I need to add a "|McDonald" on the end of that long line.

Of course, I can type ":set tw=0" to turn off wrapping, but who wants to have to do that every time? Surely vim has a way to adjust settings based on file type or location, like emacs has.

It didn't take long to find an example of Project specific settings on the vim wiki. Thank goodness for the example -- I definitely wouldn't have figured that syntax out just from reading manuals. From there, it was easy to make a few modifications and set textwidth=0 if I'm opening a file in my procmail directory:

" Set wrapping/textwidth according to file location and type
function! SetupEnvironment()
  let l:path = expand('%:p')
  if l:path =~ '/home/akkana/Procmail'
    " When editing spam filters, disable wrapping:
    setlocal textwidth=0
endfunction
autocmd! BufReadPost,BufNewFile * call SetupEnvironment()

Nice! But then I remembered other cases where I want to turn off wrapping. For instance, editing source code in cases where emacs doesn't work so well -- like remote logins over slow connections, or machines where emacs isn't even installed, or when I need to do a lot of global substitutes or repetitive operations. So I'd like to be able to turn off wrapping for source code.

I couldn't find any way to just say "all source code file types" in vim. But I can list the ones I use most often. While I was at it, I threw in a special wrap setting for mail files:

" Set wrapping/textwidth according to file location and type
function! SetupEnvironment()
  let l:path = expand('%:p')
  if l:path =~ '/home/akkana/Procmail'
    " When editing spam filters, disable wrapping:
    setlocal textwidth=0
  elseif (&ft == 'python' || &ft == 'c' || &ft == 'html' || &ft == 'php')
    setlocal textwidth=0
  elseif (&ft == 'mail')
    " Slightly narrower width for mail (and override mutt's override):
    setlocal textwidth=68
  else
    " default textwidth slightly narrower than the default
    setlocal textwidth=70
  endif
endfunction
autocmd! BufReadPost,BufNewFile * call SetupEnvironment()

As long as we're looking at language-specific settings, what about doing language-specific indentation like emacs does? I've always suspected vim must have a way to do that, but it doesn't enable it automatically like emacs does. You need to set three variables, assuming you prefer to use spaces rather than tabs:

" Indent specifically for the current filetype
filetype indent on
" Set indent level to 4, using spaces, not tabs
set expandtab shiftwidth=4

Then you can also use useful commands like << and >> for in- and out-denting blocks of code, or ==, for indenting to the right level. It turns out vim's language indenting isn't all that smart, at least for Python, and gets the wrong answer a lot of them time. You can't rely on it as a syntax checker the way you can with emacs. But it's a lot better than no language-specific indentation.

I will be a much happier vimmer now!

Tags: , ,
[ 11:29 Jun 15, 2014    More linux/editors | permalink to this entry | comments ]

Tue, 29 Apr 2014

The evil HTML double-dash problem in Emacs is still there

Long ago (in 2006!), I blogged on an annoying misfeature of Emacs when editing HTML files: you can't type double dashes. Emacs sees them as an SGML comment and insists on indenting all subsequent lines in strange ways.

I wrote about finding a fix for the problem, involving commenting out four lines in sgml-mode.el. That file had a comment at the very beginning suggesting that they know about the problem and had guarded against it, but obviously it didn't work and the variable that was supposed to control the behavior had been overridden by other hardwired behaviors.

That fix has worked well for eight years. But just lately, I've been getting a lot of annoying warnings when I edit HTML files: "Error: autoloading failed to define function sgml_lexical_context". Apparently the ancient copy of sgml-mode.el that I'd been using all these years was no longer compatible with ... something else somewhere inside emacs. I needed to update it.

Maybe, some time during the intervening 8 years, they'd actually fixed the problem? I was hopeful. I moved my old patched sgml-mode.el aside and edited some files. But the first time I tried typing a double dashes -- like this, with text inside that's long enough to wrap to a new line -- I saw that the problem wasn't fixed at all.

I got a copy of the latest sgml-mode.el -- on Debian, that meant:

apt-get install emacs23-el
cp /usr/share/emacs/23.4/lisp/textmodes/sgml-mode.el.gz ~/.emacs-lisp
gunzip ~/.emacs-lisp/sgml-mode.el.gz
Then I edited the file and started searching for strings like font-lock and comment.

Unfortunately, the solution I documented in my old blog post is no longer helpful. The code has changed too much, and now there are many, many different places where automatic comment handling happens. I had to comment out each of them bit by bit before I finally found the section that's now causing the problem. Commenting out these lines fixed it:

   (set (make-local-variable 'indent-line-function) 'sgml-indent-line)
   (set (make-local-variable 'comment-start) "")
   (set (make-local-variable 'comment-indent-function) 'sgml-comment-indent)
   (set (make-local-variable 'comment-line-break-function)
        'sgml-comment-indent-new-line)

I didn't have to remove any .elc files, like I did in 2006; just putting the sgml-mode.el file in my Emacs load-path was enough. I keep all my customized Emacs code in a directory called .emacs-lisp, and in my .emacs I make sure it's in my path:

(setq load-path (cons "~/.emacs-lisp/" load-path))
And now I can type double dashes again. Whew!

Tags: , ,
[ 12:42 Apr 29, 2014    More linux/editors | permalink to this entry | comments ]

Wed, 05 Jun 2013

Stop Emacs from invoking a browser

After upgrading my OS (in this case, to Debian sid), I noticed that my browser window kept being replaced with an HTML file I was editing in emacs. I'd hit Back or close the tab, and the next time I checked, there it was again, my HTML source.

I'm sure it's a nice feature that emacs can show me my HTML in a browser. But it's not cool to be replacing my current page without asking. How do I turn it off? A little searching revealed that this was html-autoview-mode, which apparently at some point started defaulting to ON instead of OFF. Running M-x html-autoview-mode toggles it back off for the current session -- but that's no help if I want it off every time I start emacs.

I couldn't find any documentation for this, and the obvious (html-autoview-mode nil) in .emacs didn't work -- first, it gives a syntax error because the function isn't defined until after you've loaded html-mode, but even if you put it in your html-mode hook, it still doesn't work.

I had to read the source of sgml-mode.el. (M-x describe-function html-autoview-mode also would have told me, if I had already loaded html-mode, but I didn't realize that until later.) Turns out html-autoview-mode turns off if its argument is negative, not nil. So I added it to my html derived mode:

(define-derived-mode html-wrap-mode html-mode "HTML wrap mode"
  (auto-fill-mode)
  ;; Don't call an external browser every time you save an html file:
  (html-autoview-mode -1)
)

Tags: , ,
[ 22:48 Jun 05, 2013    More linux/editors | permalink to this entry | comments ]