Category: Research

No, email is not a collaborative editing solution—A comparison of the state-of-the-art

11 May 2016

If you have co-authored at least one document in your life, you have probably experienced the pain of merging changes manually, especially when these changes are sent as email attachments.

Email is not collaborative editing — Source The Oatmeal

Nowadays there are multiple collaborative editing solutions that enable merging changes automatically. Unfortunately, many professionals are not aware of the capabilities offered by these solutions and keep sending changes as email attachments and merging them manually. Believe it or not, these professionals include software engineers and computer science researchers.

I think we can do better than that…

In this post, I aim to clarify the confusion around collaborative editing solutions by comparing the state-of-the-art:

Google Docs + Drive
Microsoft Word + OneDrive/SharePoint
Overleaf

The comparison is based on several criteria:

On-line co-authoring: the ability to edit on-line while seeing others’ changes in real-time
Off-line co-authoring: the ability to edit off-line and merge changes automatically when back on-line
Version control
WYSIWYG editor
LaTeX editor
Formatting capabilities
Support for OOXML format (DOCX, PPTX, XLSX, etc.): unfortunately, this proprietary format from Microsoft is still the most used document format
Support for OpenDocument format (ODT, ODP, ODS, etc.): this ISO standard format is used by many public institutions
Terms of service

Google Docs + Drive

Google Docs + Drive is probably the most used collaborative editing solution. It was released in 2006.

Pros

On-line co-authoring in the browser
Version tracking
WYSIWYG editor

Cons

Off-line co-authoring in Chrome only
Minimal formatting capabilities
Read-only support for OOXML format
Read-only support for OpenDocument format
Google Terms of Service

Unfortunately, the contras of Google Docs + Drive outnumber the pros.

The read-only support for OOXML and OpenDocument formats is particularly annoying. When you first edit a file in OOXML or OpenDocument format, Google Docs automatically creates a copy of the file converted into Google Drive format (GDOC, GSLIDES, GSHEET, etc.) without notifying you. For instance, this means that when you first edit a DOCX file, you will not edit this file as you would expect, but its GDOC copy, and you will end up having two copies of the same document in Google Drive. This may confuse you and your co-authors since you will be unsure about which copy of the document is the latest one. Moreover, this may compromise the formatting since format conversions are not always lossless.

Besides, the Google Terms of Service are particularly alarming. I am not a legal expert, so take my judgement of the terms of services with a grain of salt. Nevertheless, the Google Terms of Service (see the excerpt below) are most likely not suitable for co-authoring confidential documents.

“When you upload, submit, store, send or receive content to or through our Services, you give Google (and those we work with) a worldwide license to use, host, store, reproduce, modify, create derivative works (such as those resulting from translations, adaptations or other changes we make so that your content works better with our Services), communicate, publish, publicly perform, publicly display and distribute such content.”

Microsoft Word + OneDrive/SharePoint

Microsoft Word + OneDrive/Share Point is Microsoft’s answer to Google Docs + Drive. It was released together with Office 2013.

Pros

On-line co-authoring in the browser with Word online
On-line and off-line co-authoring with Word 2016 for Windows
Off-line co-authoring with Word 2013 for Windows, Word 2016 for Mac and Word for Windows Phone, Android, and iOS
Version tracking
WYSIWYG editor
Sufficient formatting capabilities
Support for OOXML format

Cons

No native support for OpenDocument format
Microsoft Services Agreement?

Microsoft Word + OneDrive/Share Point is superior to Google Docs + Drive by any criteria. I could not find any paragraph in the Microsoft Services Agreement that looks as scary as the one from the Google Terms of Service. Nevertheless, I am not willing to give Microsoft the benefit of the doubt, so I keep this agreement with a question mark in the list of contras until a legal expert proves me that it is suitable for co-authoring confidential documents.

Overleaf

Overleaf is the ultimate collaborative editing solution for LaTeX enthusiasts. It was released in 2013 (then called WriteLaTeX).

Pros

On-line co-authoring in the browser
Off-line co-authoring with Git
Version labelling
LaTeX editor
Maximal formatting capabilities

Cons

Steep learning curve for non-academics
No support for OOXML format
No support for OpenDocument format
Overleaf Terms of Service?

Similar to the Microsoft Services Agreement, I could not find any paragraph in the Overleaf Terms of Service that looks as scary as the one from the Google Terms of Service. Nevertheless, I keep these terms with a question mark in the list of contras until a legal expert proves me that they are suitable for co-authoring confidential documents.

Summary

The following table summarises the features offered by each of the compared collaborative editing solutions.

	On-line	Off-line	Versioning	WYSIWYG	LaTeX	Formatting	OOXML	Open Document	Terms
Google	✓	✗	✓	✓	✗	✗	✗	✗	✗
Microsoft	✓	✓	✓	✓	✗	✓	✓	✗	?
Overleaf	✓	✓	✓	✗	✓	✓	✗	✗	?

Is there a clear winner? I do not think so.

If you are a perfectionist in formatting and prefer working with LaTeX, Overleaf is the solution for you. If you are not too concerned with the formatting and prefer working with WYSIWYG editors, I recommend you to choose Microsoft Word + OneDrive/SharePoint.

I belong to the first category, and I co-author scientific papers using Overleaf only. However, I have to admit that Microsoft has done an excellent job with its collaborative editing solution, and I do not mind co-authoring other documents such as project deliverables in Word anymore if my colleagues ask me to.

The Chocolate-Hazelnut Spread Experiment

9 March 2016

Back in 2010, I found myself explaining why the Italian chocolate-hazelnut spread Nutella¹ is better than its Norwegian imitationcompetitor Nugatti to my colleagues at the University of Bergen. Yes, like most Italians, I often brag about Italian food.

One of my colleagues challenged me: “I bet what you want that if I give you a slice of bread with Nutella and another one with Nugatti, you will not recognise the difference.”

I answered: “I bet what you want that I will recognise the difference between Nutella and three other spreads, while blindfolded.”

A few weeks later, I ran the first chocolate-hazelnut spread test at the University of Bergen. The test aimed at verifying if it is possible to recognise chocolate-hazelnut spreads while blindfolded.

28 Oct 2010: Me tasting one of the four chocolate-hazelnut spread samples. Photo by Federico Mancini.

Here is the structure of the test:

3 participants
4 chocolate-hazelnut spreads, known to the participants
Each participant tasted each spread in random order, while not blindfolded
Each participant tasted each spread again in random order, while blindfolded
After the second taste, each participant matched the spreads in the first taste with the spreads in the second taste (e.g., the third spread in the first taste was the first spread in the second taste)

And here are the results:

The participants correctly matched the spreads 10 times out of 12 (i.e., 83,33% of the times).

Galvanised by the results, I grew a passion for chocolate-hazelnut spreads. Before the test, I thought that the chocolate-hazelnut spread was Nutella and that everything else on the market was just an imitation. After the test, I tasted more than 20 different chocolate-hazelnut spreads and completely reconsidered my opinion on Nutella and everything else on the market.

2 Oct 2015: My collection of chocolate-hazelnut spreads

My enthusiasm for chocolate-hazelnut spreads quickly became known to my friends and colleagues. Last year (i.e., five years after the first chocolate-hazelnut spread test), I was introduced to a new researcher at the University of Oslo, who told me: “Wait a minute; you must be the Nutella guy!” I was not sure if to be proud or depressed by the fact that my input to the field of chocolate-hazelnut spreads got more attention than my contribution in the field of software engineering 🙂 Nevertheless, I decided it was time to replicate the chocolate-hazelnut spread test, turning it into a full-blown experiment.

Eventually, two weeks ago, I ran the chocolate-hazelnut experiment at SINTEF. The experiment aimed at ranking the best and worst tasting chocolate-hazelnut spreads, in addition to verifying if it is possible to recognise chocolate-hazelnut spreads while blindfolded. I know, this is not the kind of experiment you would expect from a researcher in software engineering… But it is serious research nevertheless 🙂

24 Feb 2016: Me presenting the historical context of the experiment. Photo by Einar Broch Johnsen.

24 Feb 2016: Participants tasting chocolate-hazelnut spread samples. Photo by Einar Broch Johnsen.

Here is the structure of the experiment:

12 participants (9 males, 3 females) from 7 countries (4 from Norway, 3 from Italy, 1 from Bulgaria, 1 from France, 1 from Russia, 1 from Sri Lanka, 1 from Ukraine)
4 chocolate-hazelnut spreads (Merenda from Greece, Nugatti from Norway, Nutella from Italy, Venchi Crema Suprema XV from Italy), known to me only
Each participant tasted each spread in random order, while not blindfolded
After the first taste, each participant chose the best and worst tasting spreads
Each participant tasted each spread again in random order, while blindfolded
After the second taste, each participant matched the spreads in the first taste with the spreads in the second taste (e.g., the third spread in the first taste was the first spread in the second taste)

And here are the results:

7 participants ranked Venchi, 3 participants ranked Merenda, and 2 participants ranked Nutella as the best tasting spread. No participants ranked Nugatti as the best tasting spread.

8 participants ranked Nugatti, 3 participants ranked Merenda, and 1 participant ranked Venchi as the worst tasting spread. No participants ranked Nutella as the worst tasting spread.

The participants correctly matched the spreads 33 times out of 48 (i.e., 68,75% of the times).

6 participants correctly matched all spreads, 3 participants correctly matched only 2 spreads, 3 participants correctly matched only 1 spread.

3 women out of 3 correctly matched all spreads.

3 men out of 9 correctly matched all spreads.

To summarise, the participants ranked Venchi as the best and Nugatti as the worst tasting spread; they correctly matched the spreads more than two-thirds of the times; women were better than men at correctly matching the spreads.

Can we conclude that my colleague at the University of Bergen lost his bet? Can we also generalise the result, and propose a generic framework for blind tasting foods and beverages? Comments and questions are more than welcome 🙂

Special thanks to my friends and colleagues who participated in the experiment.

Update 20 June 2018

Yesterday, I had the pleasure of replicating the experiment at EVRY.

Here is the structure of the experiment:

9 participants (6 males, 3 females) from 5 countries (5 from Norway, 1 from Ireland, 1 from Italy, 1 from the Netherlands, and 1 from Sweden)
4 chocolate-hazelnut spreads (First Price Nøttepålegg from Norway, Nugatti from Norway, Nutella from Italy, Venchi Crema Suprema Fondente² from Italy), known to me only
Each participant tasted each spread in random order, while not blindfolded
After the first taste, each participant chose the best and worst tasting spreads
Each participant tasted each spread again in random order, while blindfolded
After the second taste, each participant matched the spreads in the first taste with the spreads in the second taste (e.g., the third spread in the first taste was the first spread in the second taste)

And here are the results:

3 participants ranked Nutella, 2 participants ranked Nugatti, 2 participants ranked Nøttepålegg, and 2 participants ranked Venchi Crema Suprema Fondente as the best tasting spread.

5 participants ranked Nøttepålegg, 2 participants ranked Nugatti, and 2 participants ranked Venchi Crema Suprema Fondente as the worst tasting spread. No participants ranked Nutella as the worst tasting spread.

The participants correctly matched the spreads 17 times out of 36 (i.e., 47% of the times).

2 participants correctly matched all spreads, 3 participants correctly matched only 2 spreads, 3 participants correctly matched only 1 spread, 1 participant did not correctly match any spreads.

0 women out of 3 correctly matched all spreads.

2 men out of 6 correctly matched all spreads.

To summarise, the participants ranked Nutella as the best (although it was not a clear winner) and Nøttepålegg as the worst tasting spread; they correctly matched the spreads less than half of the times (contra more than two-thirds in the 2016 experiment); men were better than women at correctly matching the spreads (contra the opposite in the 2016 experiment).

To be honest, these results are disappointing. First, although dark chocolate may not be everyone’s taste, I expected the Venchi Crema Suprema Fondente to be ranked as best. Second, I expected the participants to match the spreads correctly at least two-thirds of the times. Finally, since it is scientifically proven that women have a better sense of smell than men, I expected women to be better than men at matching the spreads correctly. But… Data is data, and these results encourage me to replicate the experiments once again in the future.

Special thanks to my friends and colleagues who participated in the experiment.

Footnotes

If you have never heard about Nutella, or if you thought Nutella was German, you should read its history. ↩
Note that the Venchi spread used in the 2018 experiment is not the same as the one used in the 2016 experiment, but the dark chocolate variant ↩

A new beginning

26 October 2012

Dear friends and colleagues,

I would like to thank you for these last five years in Bergen. They made me grow personally and professionally, and I will never forget them.

Now, however, it is time for me to move on. I have accepted a position as Research Scientist at SINTEF ICT in Oslo, and I am looking forward to starting.

My former supervisor Uwe once wrote me “Sometimes changes open unforeseen new perspectives.” This sentence has never seemed more appropriate.

See you in one month, Oslo.

The last four years of my life

8 December 2011

From the PhD thesis Diagram Predicate Framework meets Model Versioning and Deep Metamodelling, defended on the 7th December 2011:

Preface

The last four years of my life have been dedicated to writing this thesis and to making it as perfect as possible. These years have witnessed days and nights of hard work, discussion, stress, frustration, anguish, insomnia, as well as praise, relief, travelling and fun.

If you are going to read this thesis, I hope that you will find it interesting. If you are just going to browse through it quickly, I hope that you will find the models as beautiful as I do. If you are only interested in this preface, I hope it will leave you with a nice memory.

Bergen, 3rd October 2011

Acknowledgements

This thesis would not have been possible without the contribution of the outstanding individuals I have met during these four years.

First of all, I would like to thank my supervisor Uwe Wolter, for teaching me a lot of interesting knowledge which spans from mathematics to philosophy and history, as well as for giving me invaluable feedback about my research. He deserves much of the credit for this thesis, and I am indebted to him for all his help and inspiration, scientifically and otherwise. I would also like to thank my co-supervisor Khalid A. Mughal, for suggesting that I enrol in a PhD programme and for supporting all my choices when I finally followed his suggestion. With time I realised that his initiative saved me from becoming a frustrated software engineer.

A special thanks goes to Adrian Rutle, for helping me to get started with my research and for sharing many good times with me, both in Bergen and while travelling. He has been a brilliant colleague and a good friend, and I have many good memories from these years.

I am grateful to my parents Pompilio and Loretta, for all they have done for me, especially for setting my life on what I believe is the right path. I hope that this thesis will make them as proud of me as I am of them.

“Tusen takk” to Synnøve Solberg Tokerud, for her love and friendship, for teaching me about Norwegian and Norway, as well as for her beautiful smile which always helped me to stay positive.

The Department of Informatics at the University of Bergen has given me a private office, a good salary and great financial support, and I am thankful for that. I would like to thank the Programming Theory group, especially Marc Bezem, Torill Hamre, Anya Helene Bagge, Valentin David, Dag Hovland and Federico Mancini, for creating a stimulating environment to work in, for all the chats about informatics and teaching, for all the empirical studies on espresso and on chocolate spreads, as well as for all the feedback they gave me about my work. I am also grateful to the administration of the Department of Informatics, especially Ida Holen, for patiently listening to my rants every time I needed to vent my frustration, Petter Bjørstad and Torleiv Kløve, for supporting my stays abroad, and Steinar Heldal, for guiding me through the bureaucracy of the University.

My research was carried out in cooperation with fellow researchers from the Department of Computer Engineering at the Bergen University College. Thanks to Yngve Lamo, for his suggestions about how to deal with the Norwegian system, and Florian Mantz, for being an excellent flatmate and for preparing pancakes every Sunday.

Part of this thesis was written during my 4-month stay at the Department of Computer Engineering at the Autonomous University of Madrid. “Muchas gracias” to Juan de Lara and Esther Guerra, for taking care of me during my stay and for giving me plenty of insights which ended up being almost half of this thesis.

I would like to thank my opponents Reiko Heckel and Einar Broch Johnsen, for all the time they have spent reviewing this work, and Michal Walicki, for coordinating the committee. I am also grateful to all my fellow researchers and anonymous reviewers who pointed out flaws and suggested possible improvements in my research.

Despite all the time spent preparing this thesis rather than hanging out, I still have many friends left, and they should all be awarded for their patience. In Bergen, Mikal Carlsen Østensen helped me with practically everything before and after my move to Norway. Diego Fiore has been one of my closest friends, who shared countless discussions about the grotesque society we live in with me and was a perfect companion on many suffocating trips around the world. Paolo Angelelli has also been a very good friend, who contributed a lot to the discussion about how to develop an ideal society. My stay in Madrid would not have been the same without Lucia Cammalleri, Teresa Terrana and Daniele Sidoti, who treated me like a close friend since the first day we met. In Italy, my good, old friends Maura Brandimarte, Albert Marsili, Marino Di Carlo, Graziano Liberati and Angelo Di Saverio have been there every time I was back home, and I really appreciate it.

Finally, this thesis would not have reached this level of art without the free and open source software I use and enjoy. A special thanks goes to the communities behind GNU, Linux, KDE, Firefox, Kile, Inkscape, Subversion and Git.

Four months in Madrid

26 May 2011

Shipol Airport, Asterdam. Two hours left before my connection to Bergen. I am not coming back home from a business or leisure trip this time, but from a four-month exchange stay in Madrid.

Research fellows at the University of Bergen are encouraged to spend from three to six months abroad to get in touch with another research group and work in a different environment. During the MoDELS 2010 conference in Oslo, my supervisor and I discussed the possibility of my exchange stay with Juan de Lara and Eshter Guerra from the Autonomous University of Madrid. The idea of staying some months in the south of Europe after three years in the North was appealing to me, and Juan and Esther seemed positive as well. Eventually, the idea became a plan, and I came to Madrid in February.

I lived in the centre of Madrid, which happens to be the centre of Spain as well, historically speaking. My flat was located 200 meters away from the so-called Kilometre zero, the ancient starting point of all the measurements in Spain. I loved the atmosphere of the city centre, incredibly lively and dynamic. I even loved the noise that you hear in the bars… Yes, the noise of people speaking and toasting and laughing and enjoying life, something that reminded me a bit of Italy and that I missed so much in Norway, where people are usually scared of speaking too loud or too much.

Juan and Esther have been very kind to me. They helped me with the accommodation and the transportation, provided me with an office and a workstation, introduced me to the campus and the city. It has been a rewarding experience to work with them, both scientifically and personally, and I sincerely hope that we will continue the cooperation in the future.

But my stay in Madrid would not have been the same without the people I met there. Thanks to Serena, I got in touch with a group of people from Italy, France and Spain. It was a pleasure to meet Federica, Antonino, Mathilde, Vani, Ysa, Clara and Jose. But above all, it was fantastic to meet Lucia, Teresa and Daniele; lovely people, who treated me like a close friend since the first day we met. I wish most of Italians were people like them; I would consider moving back to Italy.