Shallow Thoughts : tags : vim

Key Bindings for Copy and Paste Under X11

In the previous article I wrote about how the two X selections, the primary and clipboard, work. But I glossed over the details of key bindings to copy and paste the two selections in various apps.

That's because it's complicated. Although having the two selections available is really a wonderful feature, I can understand why so many people are confused about it or think that copy and paste just generally doesn't work on Linux -- because apps are so woefully inconsistent about their key bindings, and what they do have is so poorly documented.

"Why don't they all just use the standard Ctrl-C, Ctrl-V?" you ask. (Cmd-C, Cmd-V for Mac users, but I'm not going to try to include the Mac versions of every binding in this article; Mac users will have to generalize.)

Simple: those keys have well-known and very commonly used bindings already, which date back to long before copy and paste were invented.

Read more ...

Tags: linux, X11, cmdline, emacs, vim, editors
[ 12:54 Sep 11, 2020    More linux | permalink to this entry | ]

Visual diffs and file merges with vimdiff

I needed to merge some changes from a development file into the file on the real website, and discovered that the program I most often use for that, meld, is in one of its all too frequent periods where its developers break it in ways that make it unusable for a few months. (Some of this is related to GTK, which is a whole separate rant.)

That led me to explore some other diff/merge alternatives. I've used tkdiff quite a bit for viewing diffs, but when I tried to use it to merge one file into another I found its merge just too hard to use. Likewise for emacs: it's a wonderful editor but I never did figure out how to get ediff to show diffs reliably, let alone merge from one file to another.

But vimdiff looked a lot easier and had a lot more documentation available, and actually works pretty well.

I normally run vim in an xterm window, but for a diff/merge tool, I want a very wide window which will show the diffs side by side. So I used gvimdiff instead of regular vimdiff: gvimdiff docs.dev/filename docs.production/filename

Configuring gvimdiff to see diffs

gvimdiff initially pops up a tiny little window, and it ignores Xdefaults. Of course you can resize it, but who wants to do that every time? You can control the initial size by setting the lines and columns variables in .vimrc. About 180 columns by 60 lines worked pretty well for my fonts on my monitor, showing two 80-column files side by side. But clearly I don't want to set that in .vimrc so that it runs every time I run vim; I only want that super-wide size when I'm running a side-by-side diff.

You can control that by checking the &diff variable in .vimrc:

if &diff
    set lines=58
    set columns=180
endif

If you do decide to resize the window, you'll notice that the separator between the two files doesn't stay in the center: it gives you lots of space for the right file and hardly any for the left. Inside that same &diff clause, this somewhat arcane incantation tells vim to keep the separator centered:

    autocmd VimResized * exec "normal \<C-w>="

I also found that the colors, in the vim scheme I was using, made it impossible to see highlighted text. You can go in and edit the color scheme and make your own, of course, but an easy way quick fix is to set all highlighting to one color, like yellow, inside the if $diff section:

    highlight DiffAdd    cterm=bold gui=none guibg=Yellow
    highlight DiffDelete cterm=bold gui=none guibg=Yellow
    highlight DiffChange cterm=bold gui=none guibg=Yellow
    highlight DiffText   cterm=bold gui=none guibg=Yellow

Merging changes

Okay, once you can view the differences between the two files, how do you merge from one to the other? Most online sources are quite vague on that, but it's actually fairly easy:

]c	jumps to the next difference
[c	jumps to the previous difference
dp	makes them both look like the left side (apparently stands for diff put
do	makes them both look like the right side (apparently stands for diff obtain

The only difficult part is that it's not really undoable. u (the normal vim undo keystroke) works inconsistently after dp: the focus is generally in the left window, so u applies to that window, while dp modified the right window and the undo doesn't apply there. If you put this in your .vimrc

nmap du :wincmd w<cr>:normal u<cr>:wincmd w<cr>

then you can use du to undo changes in the right window, while u still undoes in the left window. So you still have to keep track of which direction your changes are going.

Worse, neither undo nor this du command restores the highlighting showing there's a difference between the two files. So, really, undoing should be reserved for emergencies; if you try to rely on it much you'll end up being unsure what has and hasn't changed.

In the end, vimdiff probably works best for straightforward diffs, and it's probably best get in the habit of always merging from right to left, using do. In other words, run vimdiff file-to-merge-to file-to-merge-from, and think about each change before doing it to make it less likely that you'll need to undo.

And hope that whatever silly transient bug in meld drove you to use vimdiff gets fixed quickly.

Tags: linux, editors, vim
[ 20:10 Jun 09, 2016    More linux/editors | permalink to this entry | ]

Vim: Set wrapping and indentation according to file type

Although I use emacs for most of my coding, I use vim quite a lot too, for quick edits, mail messages, and anything I need to edit when logged onto a remote server. In particular, that means editing my procmail spam filter files on the mail server.

The spam rules are mostly lists of regular expression patterns, and they can include long lines, such as:
gift ?card .*(Visa|Walgreen|Applebee|Costco|Starbucks|Whitestrips|free|Wal.?mart|Arby)

My default vim settings for editing text, including line wrap, don't work if get a flood of messages offering McDonald's gift cards and decide I need to add a "|McDonald" on the end of that long line.

Of course, I can type ":set tw=0" to turn off wrapping, but who wants to have to do that every time? Surely vim has a way to adjust settings based on file type or location, like emacs has.

It didn't take long to find an example of Project specific settings on the vim wiki. Thank goodness for the example -- I definitely wouldn't have figured that syntax out just from reading manuals. From there, it was easy to make a few modifications and set textwidth=0 if I'm opening a file in my procmail directory:

" Set wrapping/textwidth according to file location and type
function! SetupEnvironment()
  let l:path = expand('%:p')
  if l:path =~ '/home/akkana/Procmail'
    " When editing spam filters, disable wrapping:
    setlocal textwidth=0
endfunction
autocmd! BufReadPost,BufNewFile * call SetupEnvironment()

Nice! But then I remembered other cases where I want to turn off wrapping. For instance, editing source code in cases where emacs doesn't work so well -- like remote logins over slow connections, or machines where emacs isn't even installed, or when I need to do a lot of global substitutes or repetitive operations. So I'd like to be able to turn off wrapping for source code.

I couldn't find any way to just say "all source code file types" in vim. But I can list the ones I use most often. While I was at it, I threw in a special wrap setting for mail files:

" Set wrapping/textwidth according to file location and type
function! SetupEnvironment()
  let l:path = expand('%:p')
  if l:path =~ '/home/akkana/Procmail'
    " When editing spam filters, disable wrapping:
    setlocal textwidth=0
  elseif (&ft == 'python' || &ft == 'c' || &ft == 'html' || &ft == 'php')
    setlocal textwidth=0
  elseif (&ft == 'mail')
    " Slightly narrower width for mail (and override mutt's override):
    setlocal textwidth=68
  else
    " default textwidth slightly narrower than the default
    setlocal textwidth=70
  endif
endfunction
autocmd! BufReadPost,BufNewFile * call SetupEnvironment()

As long as we're looking at language-specific settings, what about doing language-specific indentation like emacs does? I've always suspected vim must have a way to do that, but it doesn't enable it automatically like emacs does. You need to set three variables, assuming you prefer to use spaces rather than tabs:

" Indent specifically for the current filetype
filetype indent on
" Set indent level to 4, using spaces, not tabs
set expandtab shiftwidth=4

Then you can also use useful commands like << and >> for in- and out-denting blocks of code, or ==, for indenting to the right level. It turns out vim's language indenting isn't all that smart, at least for Python, and gets the wrong answer a lot of them time. You can't rely on it as a syntax checker the way you can with emacs. But it's a lot better than no language-specific indentation.

I will be a much happier vimmer now!

Tags: editors, vim, programming
[ 11:29 Jun 15, 2014    More linux/editors | permalink to this entry | ]

Converting C to Python with a vi regexp

I'm fiddling with a serial motor controller board, trying to get it working with a Raspberry Pi. (It works nicely with an Arduino, but one thing I'm learning is that everything hardware-related is far easier with Arduino than with RPi.)

The excellent Arduino library helpfully provided by Pololu has a list of all the commands the board understands. Since it's Arduino, they're in C++, and look something like this:

#define QIK_GET_FIRMWARE_VERSION         0x81
#define QIK_GET_ERROR_BYTE               0x82
#define QIK_GET_CONFIGURATION_PARAMETER  0x83
[ ... ]
#define QIK_CONFIG_DEVICE_ID                        0
#define QIK_CONFIG_PWM_PARAMETER                    1

and so on.

On the Arduino side, I'd prefer to use Python, so I need to get them to look more like:

    QIK_GET_FIRMWARE_VERSION = 0x81
    QIK_GET_ERROR_BYTE = 0x82
    QIK_GET_CONFIGURATION_PARAMETER = 0x83
[ ... ]
    QIK_CONFIG_DEVICE_ID = 0
    QIK_CONFIG_PWM_PARAMETER = 1

... and so on ... with an indent at the beginning of each line since I want this to be part of a class.

There are 32 #defines, so of course, I didn't want to make all those changes by hand. So I used vim. It took a little fiddling -- mostly because I'd forgotten that vim doesn't offer + to mean "one or more repetitions", so I had to use * instead. Here's the expression I ended up with:

.,$s/\#define *\([A-Z0-9_]*\) *\(.*\)/ \1 = \2/

In English, you can read this as:

From the current line to the end of the file (,.$/), look for a pattern consisting of only capital letters, digits and underscores ([A-Z0-9_]). Save that as expression #1 (\( \)). Skip over any spaces, then take the rest of the line (.*), and call it expression #2 (\( \)).

Then replace all that with a new line consisting of 4 spaces, expression 1, a spaced-out equals sign, and expression 2 ( \1 = \2).

Who knew that all you needed was a one-line regular expression to translate C into Python?

(Okay, so maybe it's not quite that simple. Too bad a regexp won't handle the logic inside the library as well, and the pin assignments.)

Tags: regexp, linux, programming, python, editors, vim
[ 21:38 Jan 19, 2013    More linux/editors | permalink to this entry | ]

Vim tip: opening lines without delay

I use vim to reply to email messages in mutt. And something I do very often is to open some new lines between quoted sections. For instance, if I'm replying to mail with a section like

> Have you noticed anything weird about squirrels lately?
> I think they're plotting something.

I might want to open up space after the first line, and say

> Have you noticed anything weird about squirrels lately?

You too? One of our local squirrels ran off with my Arduino yesterday.

> I think they're plotting something.

So I need three blank lines, with the cursor on the middle line.

In vim, typically I would place the cursor on the line before the space I want to open, and type o<RETURN><ESC>O :

o

open a line below the current line

<RETURN>

make an empty line below the quoted line

<ESC>

get out of insert mode

O

open a new line above the current line and enter insert mode there

Sounds complicated, but it's pretty fast to type o<RET><ESC>O especially when you do it many times a day.

Except vim has an odd quirk: after you type that O, you have to wait a second or two. If you start typing immediately, somehow you're not in insert mode any more, and the line you just opened, and thought you were typing in, disappears.

Well, it turns out this happens because <ESC>O is a sequence that can also happen from arrow keys. So vim has a timeout it uses to figure out whether an <ESC>O sequence was from an arrow key, or two characters typed by a real person. If the O follows too soon after the <ESC>, it figures it's an arrow key. (This is discussed in more detail in this Stack Overflow thread: Vim: Delay before 'O' opens a new line?)

It turns out there's a simple solution: set the timeout much shorter, like 100 milliseconds, in .vimrc:

set ttimeoutlen=100

Then arrow keys should still work -- unless you typically use vim over a very slow connection -- but you can type the sequence yourself (unless you type very fast) and not have it confused with an arrow key.

But really, there's a better solution. Since opening up space like this is something I do so often, why not make a vim binding for it?

map <leader>o o<CR><ESC>O

Now all I have to do is type \o (backslash, \, is the "leader" character) and I get a space opened up, with a blank line both before it and after it.

Tags: editors, vim
[ 12:20 Jul 21, 2012    More linux/editors | permalink to this entry | ]

Vim/Emacs tip: use modelines for files that need special behavior

Most of the time when I edit a text file in vim, I want lines to wrap automatically as I type, at somewhere around 70 columns wide. So I set textwidth=70 in .vimrc.

But sometimes that isn't appropriate. For instance, I have a procmail rules file where I put common spam patterns that programs like spamassassin don't seem to be able to catch. So I might have lines like:

*^Subject:.*(Ink Cartridges|Hummingbird Vine|who's who of executives|Avandia|Botox|Your Email ID|Zoosk|Best airfares on the internet|UGG Boots|police training)

... and so on -- you get the idea. I can't have lines breaking in the middle, because then the procmail rule wouldn't work. So every time I add a new phrase, I have to :set tw=0 (or one of the other umpteen ways one can tell vim not to wrap lines) first.

But you can set special behavior for one specific file by adding a special comment called a "modeline" as the first line of the file.

Procmail treats any line starting with a hash, #, as a comment, and vim recognizes # as a comment. So I can add this as the first line of the procmail file:

# vim: set tw=0:

then vim will see that and un-set that default text width I specify in .vimrc.

Vim understands most common comment styles, so it should understand lines like /* vim: set tw=0: */ and // vim: set tw=0: and ; vim: set tw=0: as well.

But to make this work I had to do one more thing: in .vimrc, I had to add

set modeline

Apparently on some versions of vim this is on by default; in others it's turned off for security reasons (someone could put an evil modeline into a file which would make your vim do something when you edited it). Definitely something to be aware of, but if you mostly edit files you created yourself on your local machine, and no one else uses your machine, it's your choice whether to worry about it.

Emacs has modelines too

Emacs has mode lines too, though it calls them Local variables lines. For instance, C++ files in Mozilla's source tend to start with:

/* -*- Mode: C++; tab-width: 2; indent-tabs-mode: nil; c-basic-offset: 2 -*- */

It's awfully handy to be able to define specific indentation style for the files within a project, making it easy for emacs users, at least, to follow your preferred coding style. If only all editors understood them!

Tags: editors, vim, emacs, tip
[ 21:26 May 25, 2011    More linux/editors | permalink to this entry | ]

Non-greedy regular expressions to clean up crappy autogenerated HTML

I maintain the websites for several clubs. No surprise there -- anyone with even a moderate knowledge of HTML, or just a lack of fear of it, invariably gets drafted into that job in any non-computer club.

In one club, the person in charge of scheduling sends out an elaborate document every three months in various formats -- Word, RTF, Excel, it's different each time. The only regularity is that it's always full of crap that makes it hard to turn it into a nice simple HTML table.

This quarter, the formats were Word and RTF. I used unrtf to turn the RTF version into HTML -- and found a horrifying page full of lines like this:

<body><font size=3></font><font size=3><br>
</font><font size=3></font><b><font size=4></font></b><b><font size=4><table border=2>
</font></b><b><font size=4><tr><td><b><font size=4><font face="Arial">Club Schedule</font></font></b><b><font size=4></font></b><b><font size=4></font></b></td>
<font size=3></font><font size=3><td><font size=3><b><font face="Arial">April 13</font></b></font><font size=3></font><font size=3><br>
</font><font size=3></font><font size=3><b></b></font></td>

I've put the actual page content in bold; the rest is just junk, mostly doing nothing, mostly not even legal HTML, that needs to be eliminated if I want the page to load and display reasonably.

I didn't want to clean up that mess by hand! So I needed some regular expressions to clean it up in an editor. I tried emacs first, but emacs makes it hard to try an expression then modify it a little when the first try doesn't work, so I switched to vim.

The key to this sort of cleanup is non-greedy regular expressions. When you have a bad tag sandwiched in the middle of a line containing other tags, you want to remove everything from the <font through the next > -- but no farther, or else you'll delete real content. If you have a line like

<td><font size=3>Hello<font> world</td>

you only want to delete through the <font>, not through the </td>.

In general, you make a regular expression non-greedy by adding a ? after the wildcard -- e.g. <font.*?>. But that doesn't work in vim. In vim, you have to use \{M,N} which matches from M to N repetitions of whatever immediately precedes it. You can also use the shortcut \{-} to mean the same thing as *? (0 or more matches) in other programs.

Using that, I built up a series of regexp substitutes to clean up that unrtf mess in vim:

:%s/<\/\{0,1}font.\{-}>//g
:%s/<b><\/b>//g
:%s/<\/b><b>//g
:%s/<\/i><i>//g
:%s/<td><\/td>/<td><br><\/td>/g
:%s/<\/\{0,1}span.\{-}>//g
:%s/<\/\{0,1}center>//g

That took care of 90% of the work, leaving me with hardly any cleanup I needed to do by hand. I'll definitely keep that list around for the next time I need to do this.

Tags: regexp, html, editors, vim
[ 23:02 Mar 29, 2010    More linux/editors | permalink to this entry | ]

Vim tip: fixing the light-background color schemes

I use a light background for my X terminals (xterm and rxvt): not white, but a light grey that I find easy on the eyes. Long ago, I spent the time to set up a custom vim color scheme that works with the light background.

But sometimes I need to run vim somewhere where I don't have access to my custom scheme. It always starts up with a lot of the words displayed in yellow, completely unreadable against a light background. :set background=light doesn't help -- the default colorscheme is already intended for a light background, yet it still uses yellow characters.

I tried all the colorschemes installed with ubuntu's vim (you can get a list of them with ls /usr/share/vim/vim71/colors). The only light-background vim schemes that don't use yellow all have their primary text color as red. Do a lot of people really want to edit red text? Maybe the same people who think that yellow is a dark color?

Curiously, it turns out that if you use one of these light color schemes on a Linux console (with its black background), the yellow text isn't yellow (which would show up fine against black), but orange (which would be better on a light background).

Mikael knew the answer:

:set t_Co=88

This tells vim to use 88-color mode instead of its default of 8, and the yellow text turns light blue. Not terrifically readable but much better than yellow. Or, instead, try

:set t_Co=256

and the yellow/light blue text turns an ugly, but readable, orange (probably the same orange as the console used).

So, vim users with dark-on-light terminal schemes: add set t_Co=256 in your .vimrc (no colon) and you'll be much happier.

Update: Pádraig Brady has a great page explaining more about terminal colour highlights, including a TERM=xterm-256color setting to get vim to use 256 colors automatically. There's also a lot of good advice there on enabling colors in other console apps.

The only catch: on Ubuntu you do have to install the ncurses-term package, which will get you xterm-256color as well as 256color variants for lots of other terminal types. Here's useful page on 256-Color XTerms in Ubuntu.

Tags: editors, vim, color, tips
[ 22:29 Mar 22, 2009    More linux/editors | permalink to this entry | ]

Helpful Vim Tip: Finding Syntax for Colors

An upgrade from woody to sarge introduced a new problem with editing mail messages in vim: Subject lines appeared in yellow, against my light grey background, so they weren't readable any more.

Vim color files have always been a mystery to me. I have one which I adapted from one of the standard color schemes, but I've never been clear what the legal identifiers are or how to find out. But I changed both places where it said "ctermfg=Yellow" to another color, and nothing changed, so this time I had to find out.

Fortunately a nice person on #vim suggested :he synID (he is short for "help", of course) which told me all I needed to know. Put the cursor on the errant line and type: :echo synIDattr(synID(line("."), col("."), 1), "name")

That told me that the Subject line was syntax class "mailSubject". So I tried (copying other lines in my color file) adding this line:

hi mailSubject term=underline ctermfg=Red guifg=Red

and now all is happy again in vim land. I wish I'd learned that synID trick a long time ago!

Tags: vim, color, editors, tips
[ 10:59 Jun 22, 2005    More linux/editors | permalink to this entry | ]

Turning off Ctrl-Space in Vim

One of those niggling problems that has plagued me for a long time: in the editor vim, if I'm typing along in insert mode and instead of a space I accidentally hit control-space, vim inserts a bunch of text I didn't want, then exits insert mode. Meanwhile I'm still merrily typing away, typing what are now vim comments which invariably end up deleting the last two paragraphs I typed then doing several more operations which end up erasing the undo buffer so I can't get those paragraphs back.

Ctrl-space inserts a null character (you can verify this by putting it in a file and running od -xc on it). I've done lots of googling in the past, but it's hard to google on strings like " " or even "space" or "null", and nobody I asked had a clue what this function was called (it turns out it re-inserts whatever the last inserted characters were) so I couldn't google on the function name.

Vim's help suggests that <Nul>, <Char-0>, or <C-^V > should do it. I tried them with map, vmap, cmap, and nmap, to no avail. I also tried <C-@> since that's a historical way of referring to the null character, googling found some references to that in vim, and that's how it displays if I type it in vim.

I finally found #vim on freenode, and asked there. Last night nobody knew, but this morning, p0g found the problem: I needed to use imap, not the map/vmap/cmap/nmap I'd been using.

So here, perserved for google posterity in case other people are plagued by this problem, is the answer:

imap <Nul> <Space>

For good measure, I also mapped the character to no-op in all the other modes as well:

map  <Nul> <Nop>
vmap <Nul> <Nop>
cmap <Nul> <Nop>
nmap <Nul> <Nop>

My current .vimrc.

Tags: vim, editors, tips
[ 11:24 Feb 17, 2005    More linux/editors | permalink to this entry | ]