Scientists has reputation of being bad programmers, who wrote code that is unreadable and therefore incomprehensible to others. I have intention to become top level programmer and followed a style guide strictly. It means I have to spent sometime in adding and removing space in the code.

To my surprise, Emacs will do it for me automatically, just by hitting the TAB and it also indents smartly, which make me conformable to write long function call and split it into multiple lines. Here's an example. Also, if I miss placed a ')' or ']' the formatting will become strange and it reminders me to check.

rainfall.subset london,
rainfall.pairs,
rainfall.dublin)

I frequently search the command history. Imaging I was produce a plot and I realised there was something miss in the data, so I go back and fix the data first, then run the ggplot command again, I press Up/Down bottom many times, or just search once/two times. M-x ggplot( will give me the most recent command I typed containing the keyword ggplot(, then I press RET to select the command, which might be ggplot(gg.df, aes(lon, lat, col = city)) + geom_line() + ...... If it is not I want, I press C-r again to choose the second most recent one and repeat until I find right one.

I am a supporter of literate statistical analysis and believe we should put code, results and discoveries together in developing models. Rstudio provides an easy to use tool for this purpose, but it does not support different R sessions, so if I need to generate a report, I have to re-run all the code from beginning, which isn't particle for me with volumes data because it will take quit long.

ESS and org-mode works really well via Babel, which is more friendly to use. I can choose to run only part of the code and have the output being inserted automatically, no need to copy/paste. Also, I can choose where to execute the code, on my local machine or the remote server, or both at the same time.

These are only the surface of ESS and there are lot more useful features like spell checking for comments and documentation templates, that makes me productive and I would recommend anyone uses R to learn ESS/Emacs. The following is my current setting.

;; Adapted with one minor change from Felipe Salazar at
;; http://www.emacswiki.org/emacs/EmacsSpeaksStatistics
(require 'ess-site)
(setq ess-ask-for-ess-directory nil) ;; start R on default folder
(setq ess-local-process-name "R")
(setq ansi-color-for-comint-mode 'filter) ;;
(setq comint-scroll-to-bottom-on-input t)
(setq comint-scroll-to-bottom-on-output t)
(setq comint-move-point-for-output t)
(setq ess-eval-visibly-p 'nowait) ;; no waiting while ess evalating
(defun my-ess-start-R ()
(interactive)
(if (not (member "*R*" (mapcar (function buffer-name) (buffer-list))))
(progn
(delete-other-windows)
(setq w1 (selected-window))
(setq w1name (buffer-name))
(setq w2 (split-window w1 nil t))
(R)
(set-window-buffer w2 "*R*")
(set-window-buffer w1 w1name))))
(defun my-ess-eval ()
(interactive)
(my-ess-start-R)
(if (and transient-mark-mode mark-active)
(call-interactively 'ess-eval-region)
(call-interactively 'ess-eval-line-and-step)))
(add-hook 'ess-mode-hook
'(lambda()
(local-set-key [(shift return)] 'my-ess-eval)))
(add-hook 'inferior-ess-mode-hook
'(lambda()
(local-set-key [C-up] 'comint-previous-input)
(local-set-key [C-down] 'comint-next-input)))
(add-hook 'ess-mode-hook
(lambda ()
(flyspell-prog-mode)
(run-hooks 'prog-mode-hook)
;; (prog-mode)
))

;; REF: http://stackoverflow.com/questions/2901198/useful-keyboard-shortcuts-and-tips-for-ess-r
;; Control and up/down arrow keys to search history with matching what you've already typed:
(define-key comint-mode-map [C-up] 'comint-previous-matching-input-from-input)
(define-key comint-mode-map [C-down] 'comint-next-matching-input-from-input)

The first time I thought I need an fancy Email is when I sent an quick model update to my colleague; I have a table like this

It was written in org-mode in which I can do the formatting quickly and nicely. But once copied over to Outlook, it looks messy, and the columns does not lineup.

| Conditioning Variable | Dependent Variable | Probability |
|---------------------------------------------------–—|
| k >= 50 | t >= 50 | 0.154 |
| k >= 50 | t >= 100 | 0.111 |
| k >= 50 | t >= 200 | 0.078 |

The correct way is to insert a table in Outlook. First, I have to export the table to a CSV file, than open it in Excel, and finally copy it over to Outlook which will recognised it as a table.

I guess the purpose of that email is to give my colleague few numbers, in a way that he can compare and gain a feeling of the model. So the format is really necessary, but the workaround is really tedious.

I have another colleague who is an HTML expert and produced an company CSS style-sheet. He was kindly customised it to match the org-export class, i.e. org-ur, org-table, org-list.

So what I did was to export to org-file as a HTML and attached it in the email so that my colleague can simply click and open it in a browser, which will gives him a nicely formatted table. But people have hundreds email per day and seems to dislike attachments.

I noticed Bernt Hansen pointed out in his famous Org Mode - Organize Your Life In Plain Text! that he use org-mime to sent HTML Email. MIME, standards for Multi-Purpose Internet Mail Extensions, is an extension to plain email and enable user to exchange rich data includes image, table, video etc.

The org-mime can parse the org file into an HTML code, in a way that the email server like Office365 or Gmail can recognise and render it with pre-defined styles.

The default style looks awful: the font, the colour, size, basically nothing is right. Recently I sent about 3-5 emails using this style, I doubt the reader will spent less time in reading and comprehend it, therefore the message is not conveyed.

But the workflow is fascinating: I call org-mime-subtree function, then I just type few email address, no need to switch to system or Outlook, everything is done in Emacs and at the exact point where the main content is generated.

So I was thinking, what if the email is look as good as the attachment? What if I can apply the style to the email, that would be looks fanatic!

I did my research, the org-mime indeed provides feature to let user to change the HTML style, two example are showed on worg. The package first generate the HTML file, and than search-and-replace a certain chunk, for example,

1
2
3
<p> 
  this is a paragraph 
</p>

will becomes something like this, depends on users specification,

1
2
3
<p style="blue">
  this is a paragraph 
</p>

The search-replace mechanics works fine, for a small email. It takes a pair value (element, style), where element can be paragraph, table, list and style can be colour, font, size etc. The problem is this pair is not quick match the standard CSS file,

1
2
3
4
5
6
7
body {
    font-family: "Helvetica Neue", "Lucida Grande", "Lucida Sans Unicode", Helvetica, Arial, sans-serif !important;
    font-size: 14px;
}
body #content {
    padding-top: 70px;
}

One can processing the CSS file, and feed the package a long list of pairs. But this approach seems not safe. I quickly skim the CSS file and found something I couldn't understand, for example the body #content block above.

I think the most problem-free approach is to follow org-export-html and ensure the generated Email has same style as exported HTML and org-MIME package will eventually implement this, but I don't to wait and decide to hack.

The script is formatted in a nice way, and looks like a textbook C program: it first declares variable and functions, with concise documentation so one can visualise the structure after reading 5-10 minutes. But the implementation is way beyond my knowldge on Emacs-Lisp language. I almost looked up each function that be used, take this snippet for example,

1
2
3
4
5
6
 
(with-temp-buffer
  (insert html)
  (goto-char (point-min))
  (run-hooks 'org-mime-html-hook)
  (buffer-string))

I have no idea of what does it means. You know the feeling when you try to learn an foreign language but took the wrong book that way above your level, and you find there no single word you could understand, and you was like What The Hell? That was my feeling.

The strategy I came up was to build up my Emacs-Lisp vocabulary: try to understand the functions/processes and translate it into a plain English, for example,

with-temp-buffer

create a temporary buffer

insert

insert the string, in this case, called html, at point.

goto-char

move the cursor, which is called point in emacs, to somewhere

point-min

means the begining of a buffer/file

run-hook

run functions that links to org-mime-html-hooks

buffer-string

return a buffer as a string

Now that I understand each words, I need to comprehense it and combine than together to understand the mean of this snippet. I tried to write in a plain English and the first attempt is like this

create temporary buffer, insert the generated html file, than move the cursor to the very start, and than apply other functions that links to org-mime-htmize

I continue this word-sentence-paragraph process and I understand few functions. But it can goes on and on, and the more I learn about Emacs lisp, the future away I digress from my original goal: apply the style sheet to HTML email. I guess this is a common dilemma in working with multi-languages. Usually I follow my interests but this time I choose to focus on achieving the goal.

It turned out it is a right decision. The concept of "inline-CSS" is mentioned int the script, I googled and found out the solution within 10 minutes. I realised that what I need to do is add a block in beginning of the HTML mail!! BINGO!

1
2
3
4
5
6
<head>
  <style>
    ...
  </style>
</head>
;; html email content starts here 

Here's the settings:

1
2
3
4
5
6
7
8
9
10
11
12
(require 'org-mime)
(add-hook 'org-mime-html-hook
          (lambda ()
            (insert
             "       
<head>
<style>
;; content of the .css file 
</style>
</head>"
             ))
          t)

I try to write as concise as possible and I use word count as a benchmark. Counting the words does not sounds like a trivial task in my cases because I have a habit to comment, even for general writing. I may comment out the whole paragraph, and leave a note aside about why, which are kept as it will be helpful in edit/review. These comments and notes should not be counted since the reader can't see them.

Addition to comments, there is a full list that does not count for technical articles, like source code, tables, figure captions etc. Some people may add reference section to the list as well.

org-wc provides the org-wc-subtree function that know what to count and what not to count. Also, org-wc-display will loop though all sections and overlay the number of words to each section headline. It is particularly useful when I need to know which sections needs to trim down and which to add more.

One of my daily achievement is to complete a writing challenge, which is about either to have write about 500 words or 45 minutes, whichever comes first. It is like a racing game for me, knowing the time or number of words is important. Tracking time is simple in Org-mode but words is problematic: I have to call the org-wc-subtree function manually. I raised a issues on GitHub and guided to nanowrimo mode, which updates the word counts while I am typing and shows it on mode-line.

It works out of box for me. The number of words is adjacent to the time I spent, which make it is very convenient to compare. Also, it calculate the average number of words per minute. It use this number to predict how long I need to achieve my daily goals (which is 500 words). The picture above shows that I spent 30 minutes editing and there are 254 words in this section.

I have a little OCD about font since university. I use Time News Rome for formal report and any other serif font for general writing because they make paragraph and text easier to read.

There was a time my friend passed me a PDF file and asked me to review it. The problem was it was in Arif font (I think) which looks terrible, and also writing became unpleasant. This experience makes me to think what is the best font for writing.

I did some research and come across the concept of variable-width font. As a programmer, I use Adobe's Source Code Pro font as default which means I face monospaced font all day. For a monospaced font, each character has same space.

While for variable-doth font, each cahracter takes width corresponding to it's shape. For example, the length of "i" is about 1 of 4th of "w". Needless to say, variable-width font is more close the nature of hand-writing. Emacs has a built-in variable-pitch-mode that could change the font.

But will it make any different to my writing? I am not sure at this moment, but I would like to have a special font that I solely use in writing. The link between the font and my write mind will gradually become firm, and eventually increase my productivity in writing.

Writing requires thinking and concentration. People have their own tips that help them to stay focus and get writing done, it may relates to a place, time or tools.

I tired many tips, like mediate before write, drink coffee, cut off internet but none of them works very well, the effects seems random. One problem I have in writing is that I jump between the sections quite often.

I tried to highlight the one sentence at a time so that I can focus on the one I am writing. I found hl-sentence package does exactly what I want. Also, I followed the author's suggestion and tweak the configuration to blur the other sentences to reduce the noise.

The current setting has two folder and helps me in a way that I can focus naturally: I don't need to force myself not looking other sentence.

The sentence highlight feature also has an big impact on my writing process by making the editing easier. One thing I want to achieve is to have proper length for each sentence/paragraph: If it is too short, I will merge it. If it is too long, I will break up into short sentences. The highlights give me a sense of the length visually which I used to get by reading or counting. To check how many sentences exactly for each paragraph, I move the cursor to end of a sentence by M-e, and then count how many flashes I have to reach the end of a paragraph.

I am fairly happy about the nanowrimo, hl-sentence and variable-pitch mode and the powerful Emacs. Thanks to all the authors who wrote the scripts, because of their quality work, many things work out of box and I am able to have an seamless integration to the current workflow. It has became more efficient and productive, and makes me believe the Emacs is the best writing tool for me.

Which program do you use for writing? which feature do you like most?