My Early Career Crisis (2014 - 2015)

When I was interviewed on the R Podcast on Feb 3, I briefly mentioned my early career crisis for the first time to the public. In this post, I’m going to write about the details. Recently I started reading the book “Principles” by Ray Dalio, and it has had huge impact on me, even though I haven’t finished reading it. In particular, he mentioned that many people were afraid of making mistakes, and when they do make mistakes, they never review or learn from the mistakes, which is really a shame, because it is such a great opportunity to improve oneself. This post is my attempt to review and analyze my mistakes, I hope I could do better in the future, and never go through the same type of crisis again.

I also wish this post could be useful to two types of readers: HRs/managers in companies who plan to hire fresh PhDs with the similar personality to me, and those fresh PhDs who plan to work in a company environment like me in late 2013. I ran out of time after I finished this post, and I’ll write another blog post in the future to summarize the explicit advice I want to give to managers and PhDs.

You’ve always got issues, Yihui

I had dinner with a few Iowa State alumni in San Diego on Feb 3rd when we were attending the RStudio Conference. Someone at the table recalled that I missed the useR! 2016 conference at Stanford due to visa issues, and my classmate Stephanie Zimmer said “You’ve always got issues, Yihui” followed by her trademark “Ha-ha-ha-ha”¹ after she heard about the incident. She said so and laughed for a reason.

When I first arrived in the US in 2009, my backpack was stolen in a restaurant in DC, and I lost a lot of things, including my money, passport, and the plaque of the John Chambers Software Award (I went to DC first because of this). This incident shocked everyone in the stats department at ISU later, and I quickly became “famous” before I arrived at ISU in Ames. The faculty and staff in the department kindly donated a good amount of cash to me. That was the reason I donated the royalties of the knitr book to the department four years later. Anyway, this incident was a tragedy to me in the beginning, but became a fond memory later.

So yeah, I’ve always got issues. Sometimes funny, and sometimes not so funny. Sometimes I make good trouble, and sometimes I make really bad trouble.

What I gained from doing a PhD

I want to start this post from my PhD life at Iowa State, because I believe a seed was deeply planted in this period (4.5 years), which eventually led to my crisis. I remember Philip Guo once said doing a PhD basically means you exchange wealth with freedom, i.e., you gain a lot of freedom, and sacrifice the potentially much higher income.

Freedom. That was exactly what I gained during my PhD training. I was very fortunate to have Di and Heike to be my advisors. They were exceptionally open-minded, and had always been supportive in both my research and my life. Di would grab and pay my bill every time we eat at a restaurant. Heike gave me a large sum of cash when I first arrived in Ames, but never cashed the check I wrote to her a few months later after I no longer had financial difficulties. They treat students like their children.

I have always been a self-motivated person, and almost always know clearly what I want to do. It happened that I was extremely interested in statistical graphics in the first two years at ISU. I spent most of my time on the cranvas package (a very cool R package for interactive graphics), and I think they were quite happy with my work, so was I. Occasionally I felt it was hard to stop working on it.

I was not completely focused on cranvas all the time, because of a few side projects, including the animation package and the community Capital of Statisitcs (COS) that I founded in 2006 to help promote statistics and R in China. In particular, I spent a huge amount of time on the latter, without telling my advisors. Both animation and COS were meaningful work, and helped lots and lots of people, but that should not justify my stealing of the PhD research time.

Things quickly became worse in late 2011, when I started working on knitr. I was so fascinated by the fact that I could eventually significantly improve Sweave, which had tortured me for a number of years. I didn’t talk to my advisors in advance, and jumped straight into knitr. From that week, I started to have a mixed feeling in my heart: I was super excited by knitr, and I felt guilty to leave cranvas behind. Every week I met with Di, I felt anxious because I procrastinated indefinitely on cranvas, which was supposed to be the topic of my PhD dissertation, whereas knitr was not on the original plan at all.

I was able to read the disappointment² from her eyes, but usually I’d try to forget the anxiety five minutes after I left her office and continue working on knitr.

As I said, my advisors were very open-minded. I’m pretty sure knitr wouldn’t be born, had I worked with other advisors. They were disappointed that I started to work on a package irrelevant to statistical graphics, but they didn’t stop me. To some extent, they sacrificed their own interest in favor of other people’s interest, and the number of “other people” turned out to be huge in the coming years.

I graduated in December 2013, and my thesis was about animation, cranvas, and knitr. By that time, my habit of procrastination had become very serious. I tried my very best to procrastinate on things that I didn’t intrinsically love, such as writing the PhD thesis. I finished the thesis only a few days before my defense. I was supposed to send the thesis to my PhD committee at least two weeks in advance, but I bet no one would read it two weeks in advance, so I only sent them a Dropbox link instead of a PDF attachment in the email, to give myself a few more days to complete it. That shows what an incredible expert I was at procrastination.

So Dr Xie got a PhD degree in statistics, with an invisible “master” degree of procrastination. From the eyes of onlookers, I might seem to be a successful PhD. I published a few papers, I wrote a few R packages, I was invited to many conferences, and gave a huge number of talks. I received the Snedecor Award from the stats department, the highest honor that a PhD student in the department could earn. But they didn’t see the shadow in my heart, neither did I (precisely speaking, I saw it but I just denied it).

I procrastinated, but didn’t waste my time

I have always been interested in psychology (in particular, all kinds of psychological biases). Most people have a tendency to justify and comfort themselves when they actually know they are doing something wrong but enjoying it at the same time. I’m no exception. I procrastinated on my PhD research, but I helped a lot of people, and I enjoyed developing knitr. Perhaps unlike other procrastinators, I could confidently say that I didn’t waste my time. I rarely spend time on entertainment (TV, games, or parties, etc). I actually work extremely hard, to a degree that you probably cannot imagine: see how many GIT commits I have pushed to GitHub, how many GitHub issues I have resolved, and how many questions I have answered in mailing lists and on Stack Overflow, and so on. The numbers are often on the magnitude of thousands.

But I wasted other people’s investment in me. Di tried to bring me to Iowa State, hoping that I could do excellent work in statistical graphics. I failed. That was only one of my several examples of wasting other people’s investment.

I did my summer internship at the AT&T Labs in 2012 with Simon Urbanek, and Fred Hutchinson Cancer Research Center in 2013 with Raphael Gottardo. I never showed up at any meetings at the AT&T Labs, and I probably met Simon four times throughout the summer (including once in the restroom). I was working on knitr and didn’t do the work he expected me to do. The last day before I left, I apologized for my inactivity, and Simon was basically speechless. Same thing happened the next year in Raphael’s lab: I never actively reported to him, and he had no idea what I was actually doing. I was still working on knitr. In both summers, I often slept all the way to 11am (as a night owl), and nobody checked where I was or what I was doing.

Yet I heard praises, all the time, which gave me further justifications of only working on things I liked. My software packages are open source, and now I call that type of open source “selfish open source”, because I was only thinking of making myself happy. One of the ways to make myself happy was to make others happy with my software. It was merely a side-effect and coincidence that these packages turned out to be useful to many other people.

My strengths and weaknesses

To analyze my personality, I need to start from my elementary school. I have almost always been an A+ student since I was a second grader. I never thought I was smart. I just studied extremely hard, partly due to poverty and partly due to the inspirations from a few of my older cousins who studied hard. In the three years of my high school, I rarely took a nap at noon, although napping was compulsory in the school. I often hid and studied in the restroom after everyone else in the dorm had slept at noon and in the night. It hurt my eyes, but more importantly, I lost the ability to ask questions and ask for help. I just try to figure out everything by myself. Till today, I still feel nervous when someone asks me “Do you have any questions for me?” Because I almost surely have no questions.³ If I do, I’ll figure them out by myself. I don’t talk much in daily conversations.

A couple of weeks ago when I checked in at the front desk of the hotel to attend rstudio::conf, the conversation was like this (which is a typical one of mine):

“Check in, please.” (Handing over my ID and credit card at the same time. I knew she would need my credit card for incidentals. I just wanted to save her a question.)

“Okay… You stay here for three nights and check out on Feb 4th, right?”

“Yes.”

“Your room was booked by your company, and your personal credit card will only be charged for incidentals.”

“Okay.”

“Do you need maps of the hotel and the city?”

“No.”

“Do you have any questions?”

“No.”

“Okay. Your room number is XXXX, and follow this way to the elevator.”

“Thank you!” (Phew! Ran off.)

I’m a little afraid of one-to-one conversations on non-technical topics. I often feel the other person is expecting me to say more, looking me in the eye, but I don’t have anything more to say, which leads to awkward silence. That is probably why I often avoid eye contact when talking to people.

I can also give an example of how I avoid asking for help. I love cooking, but I have a rule in the kitchen: I don’t want anyone to wander in the kitchen when I cook, and I always refuse help, no matter who offers to help, or how many guests I have to cook for.

Independence is both my strength and weakness. My independence is quite extreme. Somehow all my undergraduate and graduate advisors happened to have made good use of it. I attended Renmin University of China, and obtained my bachelor and master degrees there. My advisor Prof Zhao was more of an economist, and knew nothing (technical) about R. I had no interest in economics (you may feel funny why we chose each other, and that is another story). Most of his students worked on his projects, but I rarely did. He gave me the full freedom to learn R and statistical graphics by myself because he knew I loved them, and rarely asked me to help with his projects, although he had a lot. He offered me the financial support to travel to Germany twice to attend the useR! conference, and a statistical graphics workshop. These were my first international trips, and I met Di and Heike at the workshop for the first time. The consequences of him respecting my independence turned out to be:

1. I became a first-generation R user in China, and probably also contributed a little bit to the role of statistical computing and graphics in statistical education in China.

2. I founded COS, which included the earliest web forum for R in China (which is still alive after 12 years and I’m still active in it).

3. I organized the first R conference in China after I came back from the useR! conference in 2008. Now this conference attracts thousands of attendees every year (and it has always been free).

4. I wrote my first R package, animation. Because of this package, I got Hadley and Di’s attention. Because of their attention, I was accepted and invited to the graphics workshop. Because of this workshop, I went to Iowa State… You can complete the rest of the circle by yourself.

These things show another of my strength: if I’m allowed to work completely independently, I can pour an infinite amount of energy into things I love, and I can keep the energy for years and years. Sometimes I’m just too excited to sleep and can even keep working until I’m physically sick. However, my weaknesses are also revealed at the same time:

1. No matter if I’m allowed to work independently, I tend to work independently anyway. What’s worse, I avoid communication, because I don’t want to be interrupted, which is selfish.

2. While I’m exceptionally good at focusing on things I love for a really long time, I’m also exceptionally bad at moving my focus to the work I don’t enjoy, and I’ll procrastinate on such work.

Both weaknesses are terrible in a company environment. In the academic environment, they do not really hurt due to several reasons, including (1) there is a lot of freedom in academic research; (2) professors are often nice, kind, and busy; (3) one advisor often has multiple students working on the same project; (4) there are no strict criteria to evaluate the output of academic research (papers might be one, but professors are so good at publishing papers…); (5) sometimes academic researchers have no direct clients to serve, which means nobody cares if you succeed or fail. In general, I feel the pressure in academia is less than that in the industry.

I think the free atmosphere in academia makes sense, but it was not good for me, because I could easily be spoiled. I would be gradually sliding into a bottomless hole in this environment. I need a tough coach to guide me on how to use my energy. I felt like I had a machine gun in my hands, but I had been shooting at random objects all day long.

Joining RStudio

I made JJ’s (CEO of RStudio) acquaintance in 2012 because of the knitr package. Later we did four talks together. I liked and admired him. He was both visionary and practical. Of course, I also loved the RStudio IDE. The total time I spent on job hunting was probably five minutes: we had a brief phone call in 2013, and he made an offer. It was that simple. The salary in the offer was way beyond my expectation. I highlighted the salary here because I’ll mention it again later. It will be one of the two solutions to my problems.

The other great advantage of working for RStudio was that I was able to work from home, which means, freedom. I had already been spoiled by freedom in academia, and then I was offered even more freedom. There wouldn’t a boss in the back watching over me when I work. How free was that!

I was first assigned to the Shiny team, and worked with Joe Cheng (CTO of RStudio). My first task was to bring DataTables to Shiny. I was not a JavaScript professional and sometimes had to “copy and paste from Stack Overflow”, but I enjoyed the work, and was excited when I saw the beautiful tables rendered in Shiny apps. I have always been fascinated by web technologies.

It was a good start. Then I was interrupted by my PhD defense. I spent three weeks on my thesis, and defended it. I was still fully paid by RStudio in this period, although I didn’t deserve it. Not long after my defense, I was interrupted again by immigration issues (waiting for work authorization since I was no longer a student).

When I came back to work in early 2014, I was assigned more tasks related to JavaScript libraries, such as selectize.js. I was also expected to maintain the shiny package, since Joe became busy with other things. I was quickly overwhelmed by the number of questions in the mailing list as well as the number of GitHub issues, thanks to the popularity of Shiny. They were a setback to me. After I tried a while to answer every single question that I was able to answer, and fix bugs that I could fix, I felt frustrated. Basically I had little time left to develop anything new. I saw questions and issues every morning, which could keep me busy until midnight. An unfortunate fact about me is that I actually like debugging and fixing bugs, and I’d rather do these things instead of developing new features in Shiny.

I lost my passion, and started to step to the wrong direction, without talking to anyone. I had a few interesting projects of my own (including knitr), and I started to spend more time on them instead of my official job. In fact, these “interesting projects” included an early prototype of blogdown, but no one in the company knew I was so interested in it, because I didn’t tell them. Again, my problem of lack of communication.

Joe had discovered my procrastination before he left. He didn’t blame me but tried to save me. He introduced techniques like Pomodoro to me to fight against procrastination. I didn’t really listen. I would only work a little bit when I saw he was obviously unhappy from his face. Eventually he left me alone. And I left my job behind. The mixed feeling of anxiety and excitement started to strike me again.

In September 2014, Tareef (president of RStudio) talked to me about my problems, and gently reminded me that I should be more active at communication and more focused on my work. I felt very anxious when he talked to me, but again, five minutes later, I tried to forget it.

Perhaps they felt I was not highly motivated by Shiny, and it happened that Joe had another project, leaflet, which he started quite a while ago but didn’t have time to turn it into a formal R package. It was handed to me. I was only moderately interested in it, but it felt better than the Shiny giant. At about the same time, I also started working on DT. I favored DT because I was more familiar with the JS library DataTables. I hadn’t used the JS library leaflet.js before, and it seemed to be complicated to me. Therefore I procrastinated on leaflet but made better progress on DT. Again, I chose to work only on things I liked.

The 2015 winter vacation

I went back to China to visit my family in Jan 2015, and flew back to the US in early March. It was a vacation of two months, which was unusual (to my knowledge, most other people would take at most one week’s vacation), so I promised I would work half-time while I was in China. I failed. I think the total amount of time I spent on my work was probably twenty hours in the two months. A number of reasons why I didn’t (actively) work:

1. I went back to celebrate the Chinese new year with my family. I hadn’t had a chance in five years, because the dates for the Chinese new year are often not in the usual US winter vacation time. The Spring Festival means one thing to many Chinese: eating. I have a lot of relatives, and had to visit them one by one. Each would take a whole day. The Spring festival is much longer than Christmas in the western world (more than two weeks). I just kept eating, and felt like I was to eat all the delicious food I missed in the five years.

2. It was freezing in winter in my hometown, thanks to whoever the genius it was to decide that the southern part of China didn’t need heating in winter a few decades ago (so we don’t have the infrastructure for heating). The temperatures outside and inside the rooms are pretty much the same. I had been spoiled by the heat in Beijing for seven years, and US for five years. I had never felt the keyboard (especially the trackpad) of my Macbook was so cold. It was like typing on a block of ice. Joe joked that I should try to build RStudio from source from time to time to keep my laptop warm. In my hometown, we typically burn wood in a special room to stay warm, but there was no Internet in that room in my parents’ house.

3. Very limited Internet access. My parents live in the rural area, as most of my relatives do. Without Internet access, I gradually disappeared from the company, and nobody knew where I was or what/whether I was working (again, like when I did my internship).

After I had practically disappeared for several weeks, the company decided to change my role and cut my salary after some (presumably long) internal discussions. They thought I was not super interested in software development but teaching instead (or at least the software engineer’s role was obviously not appropriate for me). That was a misunderstanding, but completely due to my lack of communication. I love both software development and teaching. My performance as a software engineer was extremely disappointing. While I might well have the skills as a software engineer (although my skills were definitely not extraordinary), there was no sign to show that I could be accountable. I failed on shiny. I failed on leaflet. I failed on DT. The company tried really hard to save me, but they could not even find me in the first place.

Even at this point, I was not fired. Someone in the company probably still had the faith that I could be changed. In retrospect, you can imagine how grateful I am to RStudio.

The failed ICSA bulletin editor

I want to digress to one thing irrelevant to my work for a bit, because it also seriously affected me mentally in the same time period. In 2014, I accepted an invite from ICSA (International Chinese Statistical Association) and agreed to take the role of its bulletin editor, thanks to the recommendation of the previous editor. I accepted it because:

1. I was bad at saying “No”. I didn’t know how to refuse people. You can observe the same pattern in my software development: I tend to answer every single question, implement every single feature request, and fix every single bug. I don’t want to let people or users down. I didn’t know that (1) they would not necessarily be let down if I refuse to help, and (2) I had to let someone down because I had my own personal limit (I could not work 48 hours a day). Oftentimes, when I try not to disappoint one person, I’ll eventually disappoint another person, and actually disappoint that person much more badly. In this case, I tried not to let the previous editor down, and it turned out that I not only let him down, but also seriously let the president of ICSA down, and created an enormous amount of trouble to this association.

2. I was actually interested in this role, and I was very ambitious. I helped the previous editor for two years, and thought the process of editing this bulletin was too inefficient. The editor had to do a lot of work in LaTeX, which could be significantly simplified via Markdown. I wanted to make it much easier to for ICSA members to contribute to this bulletin and for the editor to put together the articles in the final PDF. I saw it cost a lot of money for ICSA to print the bulletins, which didn’t make much sense to me and I thought the money collected from members was largely wasted. I was in favor of electronic distributions, and was hoping to turn the bulletins into appealing web pages, which was possible with Markdown (so was PDF).

It turned out I was too ambitious. I had a full-time job, on which I had already been procrastinating. I seriously underestimated the effort required to change the format of the bulletin from raw LaTeX to Markdown. What I was envisioning was essentially bookdown, which didn’t exist in 2014, but I had been thinking about it for long. How long did I actually spend on the development of bookdown later? A whole year (as my full-time job). I thought I would be able to turn the bulletin source format to Markdown in about a week. You see how ridiculously optimistic I was. Even now I still don’t think I’m ready to build a bulletin with Markdown, since there are still missing pieces in bookdown to support this type of publications.

I was expected to make the bulletin ready for printing in Jan 2015. By Jan 2015, how much progress had I made? Basically zero. I tried Markdown on one article, and it didn’t seem to be straightforward. Then I left it behind. At that time, I had already got a lot of anxiety from my daily job, and I was fairly weak mentally. It was easy to give up extra challenges in this case.

Then I had the aforementioned terrible vacation, in which both RStudio and ICSA were desperately looking for me. The ICSA president tried to reach me via several approaches: phone calls, voice/text messages, emails, wechat, asking other people to call me, and so on. I was too nervous to talk to him, so I ignored these messages as an ostrich.

Finally he gave up, and appointed a new editor. That issue of the bulletin was delayed by four months, and note that it is a biannual bulletin. This probably had never happened before in the history of ICSA. I bet these ICSA people would never want to talk to me again. I felt extremely sorry and shameful about it. I still do, and will do.

Empathy vs reality

Empathy is also both a strength and a weakness of mine. In particular, I have a strong sympathy for people who suffer from inefficient or buggy software (of mine or others). It is the most important reason why I was able to create a few user-friendly software packages. As I said several times before, I don’t have super strong skills for software development (I don’t know C, C++, Java, Python, or most languages you can imagine).

It was the empathy that helped me succeed. It was also the empathy that destroyed me.

When I agreed to be the ICSA bulletin editor, I had good intentions. I very much wanted to free future editors from the laborious work. I just ignored the reality that I wouldn’t be able to make it in such a short time.

When you create a popular software package, you will receive a lot of praises, but at the same time, you will also receive a lot of feature requests, questions, and bug reports. My mistakes were (1) I was too satisfied and proud with the good words, and (2) I failed to realize my physical limit. I was lost in a forest of endless questions, requests, and reports.

Sometimes I don’t understand how (most) other software developers manage to put these issues aside. As I mentioned in a previous post, I had a ridiculous OCD that I didn’t want the number of GitHub issues to exceed 25 in any repository that I maintain. It was partly because of empathy, and partly because of my intolerance of imperfection. So far I have only observed one software developer who behaves like me: my colleague Kevin Ushey. He has been amazingly responsive in GitHub issues (I don’t know what his thoughts are, though). For most other (highly prolific) software developers that I know, I often see pages after pages of GitHub issues opened long time ago in their repositories. Perhaps that is the path that I should follow, too.

The (inflated) ego

Along with my long-time “success” came my inflated ego. Ray Dalio also mentioned this problem in “Principles”: the ego and blind spots are the two deadly flaws of most people (especially “smart people”), and it is also extremely difficult to overcome them.

I haven’t had many failures in my life, and have always been surrounded by praises. This may sound great, but it created a stupid ego, and made me so afraid of failures. Whatever I do, I want to be very successful. I could not accept failures or a self that didn’t sound as perfect as described (or imagined) by other people. It would be hard to imagine three years ago that I would write such a post to describe myself as once such a terrible badass.

The ego also made me ignore people I shouldn’t have ignored. When Joe first reminded me that I should focus more on my job, I ignored him partly because I thought I was doing awesome work (secretly). That was arrogant. Joe was so much more knowledgeable and professional than me.

Now I’m ready to kill my ego. To me, the ego will grow by itself unless I intentionally suppress it. I need to control myself and try not to seek for the mental reward from praises. Ray Dalio suggests us pay close attention to what is right or wrong in other people’s comments, instead of what makes us feel good or bad. We need to be careful enough not to be trapped by the feelings produced by our “lower-level” selves.

This explains why I no longer tweet actively and frequently (now I primarily use Twitter for replying people). I don’t want to be fooled by the big number of followers or the numbers of likes or retweets of my tweets. I have quit my blog from the popular R-bloggers (for reasons including this one) and, if I remember correctly, never actively tweeted a single blog post of mine (sometimes I mention certain posts in replies due to the character limit). It also explains why I aggressively cut my own reputation points on Stack Overflow. I want to restore my sanity and evaluate myself objectively in a quiet world.

To clarify, I definitely don’t mean people should not praise me (please don’t feel bad if you have ever praised me). It was all my own fault to indulge myself in the praises and reputation. I thought I was awesome even when I was essentially rotten inside.

If you cannot kill yourself, don’t expect to rebuild yourself.

The direct cure

In the spring of 2016, I was invited to give a talk at a conference organized by the ASA Chicago chapter, where I met Karl Broman and a few other speakers. At the dinner, I told Karl that I had got the cure for my procrastination. He seemed to find it unbelievable (perhaps because procrastination was so common in academia, and he was also a person poor at saying “No”).

A cure for procrastination? Is there such a thing in the world?!

I had read a lot of articles on how to stop procrastination. I had tried tools like Pomodoro. I can tell you that none worked for me. Some methods were actually harmful to me and made my situation worse, such as The Art of Procrastination.

What cured me eventually was, as shameful as it sounds, my reduced salary. As I mentioned earlier, JJ offered me a great number initially. I probably had the highest (initial) salary among all PhDs in statistics in the US. After I returned from the 2015 vacation, my salary was substantially reduced. The worst feeling in this world is probably when you go from “top” to “bottom” (e.g., healthy to sick, rich to poor, famous to unknown, and lovestruck to parted, etc). The opposite way feels much better.

Needless to say, I felt pretty bad at it. Fortunately, I hadn’t lost all my sanity. I thought and rethought about the whole thing, and concluded that it was completely my fault. I missed too many chances when there were helping hands in front of me. I rudely refused the help, and stubbornly went my own way, vanishing into the darkness.

Now I feel the fundamental reason why I could not stop procrastination was because procrastination didn’t really hurt (enough). I procrastinated on cranvas because it didn’t hurt me. I was still able to survive and complete my PhD. Had my advisors said “I wouldn’t let you graduate unless you resume the work on cranvas”, I’d definitely give up knitr and COS (at least temporarily).

Simply put, you just won’t procrastinate when you realize you are going to starve soon.

The fundamental cure

That was the direct cure, and it was not the best or fundamental cure. Again, I’m a self-motivated person and an introvert. Changing my personality in a short time will be hard. I do have my strengths, and the question is how to better take advantage of them.

The answer turned out to be fairly simple. I just tell the company what I want to do, and they will think if the interest of the company is aligned with mine. Guess what? They have rarely said “No” to me. They value and respect my personal interests very much.

Problem solved — if only I could talk to them.

Before that, I was very afraid of telling them my thoughts, because I was afraid of rejections. I’d imagine them saying “Hey Yihui, you shall work on Shiny since Joe is so busy”, and “There is no point of working on R Markdown since JJ is on it.” These imaginary conversations may sound funny, but they show how poor I was at communication.

My deep depression continued until I started working on bookdown in late 2015. It was a project that I wished to work on for at least two years. It was not only a project that I loved, but also a project in which Tareef patiently coached me to become a professional software engineer (that was why I thanked him in both the bookdown book and the blogdown book). The most important lesson I learned was: take some serious time to plan ahead, and try your best to stick with the plan in case of all kinds of distractions. This is drastically different with developing “selfish open-source software”. With the latter, you just do whatever you want, and whenever you like. No commitment, no responsibilities, no guarantee. While the freedom could make the developer fairly efficient sometimes (no pressure), it is harmful in the long run. You will be at best successful with a handful of things in a rather narrow field. Once you move outside of your comfort zone, your life will be utterly miserable.

Another benefit of making a plan in advance is that you will be better at estimating the time to finish a project. Besides all weaknesses I mentioned above, I have yet another one: I’m bad at estimating time. More often than not, I need twice as much time as I originally imagined. For example, when I tell Tareef I’ll finish something in two weeks, it often ends up taking me four weeks or even more.

To be honest, I’m still not comfortable with making plans and writing them down. I feel I’m busy enough every day, and don’t want to think what I want to do tomorrow, next week, or next year. But I’m still trying to think about it even only for one or two minutes every night before I go to bed. It is very helpful. In particular, it helps a lot with pushing away things that are not important but look urgent. You can always compare your plan with those “urgent” things, and decide the priorities.

Among the many emails that Tareef sent to help me in early 2015, he once quoted Chris Hadfield (the astronaut):

“Decide in your heart of hearts what really excites you and challenges you, and start moving your life in that direction. Every decision you make, from what you eat to what you do with your time tonight, turns you into who you are tomorrow, and the day after that. Look at who you want to be, and start sculpting yourself into that person. You may not get exactly where you thought you’d be, but you will be doing things that suit you in a profession you believe in. Don’t let life randomly kick you into the adult you don’t want to become.”

Don’t let life randomly kick you into the adult you don’t want to become.

I think Tareef’s diagnostics of me were very accurate. I was basically letting life randomly kick me around. I had my strong passion (for software), and I had my life goals, but I constantly drowned in the random world.

2016, bookdown, made a plan, and done. 2017, blogdown, made a plan, and done.

It seems I have gained better control over my life now, but I clearly know that there is no cure that works once and for all. I still have to reflect on my failures and mistakes from time to time, because it is just so very easy and tempting to start drifting again.

The tumor

I’m going to digress a bit again here, to describe an incident in Oct 2015. Remember, “Yihui has always got issues.” One day I suddenly had a pain in the back neck. Initially I thought it was due to an injury when I lifted a heavy object upstairs a few days ago. In the following days, the pain became severer and severer. I had never had so much pain in my body. I went to see a doctor (for the first time in my five years in the US — I had been very healthy). A couple of weeks later, the MRI showed a tumor in my neck.

Long story short, I had a surgery in December (first time in my life) and the tumor turned out to be benign. Thankfully, everything seemed to be fine. I wasn’t too scared when the tumor was first discovered, but I did think about life and death.⁴ It would be a shame if I died when I was still a terrible person (on my tombstone they would write “here lies a guy, irresponsible, and a failure”). Although it sounds like a cliche, you will be better at sorting out priorities today if you are going to die tomorrow. You wouldn’t care if “someone is WRONG on the Internet”, or someone is not convinced by your work, or someone wants a figure caption to be bold.

But I still forget this from time to time. For example, why should I care about someone who didn’t buy the convenience of roxygen2 a few weeks ago? I really shouldn’t have replied to that email. It was completely a mistake.

The future

I want to become a professional and responsible software engineer, and more importantly, I wish I won’t have to struggle much to work on my less favorite projects (such as DT). I’ll make fewer promises to the random world, because my past records show that I often failed to keep them, and the failures made me feel very guilty. I hate that feeling.

I have been working 100% on open source packages since I joined RStudio. Although I’m definitely biased from the inside view, I have to say RStudio’s commitment to open source is amazing, considering that RStudio is a relatively small company. Both RStudio and I believe firmly in open source. I feel extremely proud of working for such a company. At RStudio, I can be rest assured that I don’t need to worry about other issues as long as I can focus on my work. There is zero bureaucracy in the company. I only need to spend about 10 minutes on a weekly meeting, and no time on traffic (since I work from home). Work hard, deliver results as planned, and I’ll be paid well. It is just that simple. Isn’t it a dream company? It is hard to imagine that I was so blind back in the early days. I need to value the opportunities.

Starting an open source project is easy, but building it into a mature product is much harder. I think many people have discovered this fact, such as the very creative Romain Francois (he said this in an R podcast). As a matter of fact, Wei Zheng in the Tang dynasty of China also said it well 1500 years ago: “善始者实繁，克终者盖寡” (many people can start but few can finish).⁵ I wish I could develop a more solid habit of consistently carrying a product all the way to its destination.

I want to quote Ray Dalio again to finalize this post. I went through a long period of pain and made bad mistakes, but the wise thing to do is embrace pain and mistakes:

“Instead of feeling frustrated or overwhelmed, I saw pain as nature’s reminder that there is something important for me to learn.”

“The most valuable habit I’ve acquired is using pain to trigger quality reflections.”

“Everyone makes mistakes. The main difference is that successful people learn from them and unsuccessful people don’t.”

Thanks, Tareef, JJ, and Joe.

Update on 2018-02-17

After I published this post, I received a lot of feedback in a short period of time, and I truly appreciate all your thoughts, encouragements, and care. I will see if I could summarize them in the future. For now, I just want to clarify it to those who were concerned about my mental health issues: I’m no longer depressed. The years 2016 and 2017 have been fantastic, and I have mostly recovered. Thanks again!