Scientists has reputation of being bad programmers, who wrote code that is unreadable and therefore incomprehensible to others. I have intention to become top level programmer and followed a style guide strictly. It means I have to spent sometime in adding and removing space in the code.

To my surprise, Emacs will do it for me automatically, just by hitting the TAB and it also indents smartly, which make me conformable to write long function call and split it into multiple lines. Here's an example. Also, if I miss placed a ')' or ']' the formatting will become strange and it reminders me to check.

rainfall.subset london,
rainfall.pairs,
rainfall.dublin)

I frequently search the command history. Imaging I was produce a plot and I realised there was something miss in the data, so I go back and fix the data first, then run the ggplot command again, I press Up/Down bottom many times, or just search once/two times. M-x ggplot( will give me the most recent command I typed containing the keyword ggplot(, then I press RET to select the command, which might be ggplot(gg.df, aes(lon, lat, col = city)) + geom_line() + ...... If it is not I want, I press C-r again to choose the second most recent one and repeat until I find right one.

I am a supporter of literate statistical analysis and believe we should put code, results and discoveries together in developing models. Rstudio provides an easy to use tool for this purpose, but it does not support different R sessions, so if I need to generate a report, I have to re-run all the code from beginning, which isn't particle for me with volumes data because it will take quit long.

ESS and org-mode works really well via Babel, which is more friendly to use. I can choose to run only part of the code and have the output being inserted automatically, no need to copy/paste. Also, I can choose where to execute the code, on my local machine or the remote server, or both at the same time.

These are only the surface of ESS and there are lot more useful features like spell checking for comments and documentation templates, that makes me productive and I would recommend anyone uses R to learn ESS/Emacs. The following is my current setting.

;; Adapted with one minor change from Felipe Salazar at
;; http://www.emacswiki.org/emacs/EmacsSpeaksStatistics
(require 'ess-site)
(setq ess-ask-for-ess-directory nil) ;; start R on default folder
(setq ess-local-process-name "R")
(setq ansi-color-for-comint-mode 'filter) ;;
(setq comint-scroll-to-bottom-on-input t)
(setq comint-scroll-to-bottom-on-output t)
(setq comint-move-point-for-output t)
(setq ess-eval-visibly-p 'nowait) ;; no waiting while ess evalating
(defun my-ess-start-R ()
(interactive)
(if (not (member "*R*" (mapcar (function buffer-name) (buffer-list))))
(progn
(delete-other-windows)
(setq w1 (selected-window))
(setq w1name (buffer-name))
(setq w2 (split-window w1 nil t))
(R)
(set-window-buffer w2 "*R*")
(set-window-buffer w1 w1name))))
(defun my-ess-eval ()
(interactive)
(my-ess-start-R)
(if (and transient-mark-mode mark-active)
(call-interactively 'ess-eval-region)
(call-interactively 'ess-eval-line-and-step)))
(add-hook 'ess-mode-hook
'(lambda()
(local-set-key [(shift return)] 'my-ess-eval)))
(add-hook 'inferior-ess-mode-hook
'(lambda()
(local-set-key [C-up] 'comint-previous-input)
(local-set-key [C-down] 'comint-next-input)))
(add-hook 'ess-mode-hook
(lambda ()
(flyspell-prog-mode)
(run-hooks 'prog-mode-hook)
;; (prog-mode)
))

;; REF: http://stackoverflow.com/questions/2901198/useful-keyboard-shortcuts-and-tips-for-ess-r
;; Control and up/down arrow keys to search history with matching what you've already typed:
(define-key comint-mode-map [C-up] 'comint-previous-matching-input-from-input)
(define-key comint-mode-map [C-down] 'comint-next-matching-input-from-input)

The first time I thought I need an fancy Email is when I sent an quick model update to my colleague; I have a table like this

It was written in org-mode in which I can do the formatting quickly and nicely. But once copied over to Outlook, it looks messy, and the columns does not lineup.

| Conditioning Variable | Dependent Variable | Probability |
|---------------------------------------------------–—|
| k >= 50 | t >= 50 | 0.154 |
| k >= 50 | t >= 100 | 0.111 |
| k >= 50 | t >= 200 | 0.078 |

The correct way is to insert a table in Outlook. First, I have to export the table to a CSV file, than open it in Excel, and finally copy it over to Outlook which will recognised it as a table.

I guess the purpose of that email is to give my colleague few numbers, in a way that he can compare and gain a feeling of the model. So the format is really necessary, but the workaround is really tedious.

I have another colleague who is an HTML expert and produced an company CSS style-sheet. He was kindly customised it to match the org-export class, i.e. org-ur, org-table, org-list.

So what I did was to export to org-file as a HTML and attached it in the email so that my colleague can simply click and open it in a browser, which will gives him a nicely formatted table. But people have hundreds email per day and seems to dislike attachments.

I noticed Bernt Hansen pointed out in his famous Org Mode - Organize Your Life In Plain Text! that he use org-mime to sent HTML Email. MIME, standards for Multi-Purpose Internet Mail Extensions, is an extension to plain email and enable user to exchange rich data includes image, table, video etc.

The org-mime can parse the org file into an HTML code, in a way that the email server like Office365 or Gmail can recognise and render it with pre-defined styles.

The default style looks awful: the font, the colour, size, basically nothing is right. Recently I sent about 3-5 emails using this style, I doubt the reader will spent less time in reading and comprehend it, therefore the message is not conveyed.

But the workflow is fascinating: I call org-mime-subtree function, then I just type few email address, no need to switch to system or Outlook, everything is done in Emacs and at the exact point where the main content is generated.

So I was thinking, what if the email is look as good as the attachment? What if I can apply the style to the email, that would be looks fanatic!

I did my research, the org-mime indeed provides feature to let user to change the HTML style, two example are showed on worg. The package first generate the HTML file, and than search-and-replace a certain chunk, for example,

1
2
3
<p> 
  this is a paragraph 
</p>

will becomes something like this, depends on users specification,

1
2
3
<p style="blue">
  this is a paragraph 
</p>

The search-replace mechanics works fine, for a small email. It takes a pair value (element, style), where element can be paragraph, table, list and style can be colour, font, size etc. The problem is this pair is not quick match the standard CSS file,

1
2
3
4
5
6
7
body {
    font-family: "Helvetica Neue", "Lucida Grande", "Lucida Sans Unicode", Helvetica, Arial, sans-serif !important;
    font-size: 14px;
}
body #content {
    padding-top: 70px;
}

One can processing the CSS file, and feed the package a long list of pairs. But this approach seems not safe. I quickly skim the CSS file and found something I couldn't understand, for example the body #content block above.

I think the most problem-free approach is to follow org-export-html and ensure the generated Email has same style as exported HTML and org-MIME package will eventually implement this, but I don't to wait and decide to hack.

The script is formatted in a nice way, and looks like a textbook C program: it first declares variable and functions, with concise documentation so one can visualise the structure after reading 5-10 minutes. But the implementation is way beyond my knowldge on Emacs-Lisp language. I almost looked up each function that be used, take this snippet for example,

1
2
3
4
5
6
 
(with-temp-buffer
  (insert html)
  (goto-char (point-min))
  (run-hooks 'org-mime-html-hook)
  (buffer-string))

I have no idea of what does it means. You know the feeling when you try to learn an foreign language but took the wrong book that way above your level, and you find there no single word you could understand, and you was like What The Hell? That was my feeling.

The strategy I came up was to build up my Emacs-Lisp vocabulary: try to understand the functions/processes and translate it into a plain English, for example,

with-temp-buffer

create a temporary buffer

insert

insert the string, in this case, called html, at point.

goto-char

move the cursor, which is called point in emacs, to somewhere

point-min

means the begining of a buffer/file

run-hook

run functions that links to org-mime-html-hooks

buffer-string

return a buffer as a string

Now that I understand each words, I need to comprehense it and combine than together to understand the mean of this snippet. I tried to write in a plain English and the first attempt is like this

create temporary buffer, insert the generated html file, than move the cursor to the very start, and than apply other functions that links to org-mime-htmize

I continue this word-sentence-paragraph process and I understand few functions. But it can goes on and on, and the more I learn about Emacs lisp, the future away I digress from my original goal: apply the style sheet to HTML email. I guess this is a common dilemma in working with multi-languages. Usually I follow my interests but this time I choose to focus on achieving the goal.

It turned out it is a right decision. The concept of "inline-CSS" is mentioned int the script, I googled and found out the solution within 10 minutes. I realised that what I need to do is add a block in beginning of the HTML mail!! BINGO!

1
2
3
4
5
6
<head>
  <style>
    ...
  </style>
</head>
;; html email content starts here 

Here's the settings:

1
2
3
4
5
6
7
8
9
10
11
12
(require 'org-mime)
(add-hook 'org-mime-html-hook
          (lambda ()
            (insert
             "       
<head>
<style>
;; content of the .css file 
</style>
</head>"
             ))
          t)