Jekyll2018-04-07T22:42:49+00:00https://virviil.github.io//Virviil’s development blogThis blog is a collection of articles in software development.Elixir: Do you have HTTP requests? You are doing them wrong!2018-04-06T00:00:00+00:002018-04-06T00:00:00+00:00https://virviil.github.io/2018/04/06/elixir-do-you-have-http-requests-you-are-doing-them-wrong<p>The process of making HTTP requests in Elixir seems to be obvious for every developer:
one should take <a href="https://hex.pm/packages/httpoison"><strong>HTTPoison</strong></a> (3,5M downloads from <strong>Hex.pm</strong>), and do whatever he wants!
But… Have you ever thought about alternatives? If not - follow this article,
and get the answer to the question:</p>
<blockquote>
<p>How to make HTTP request?
<!--more--></p>
</blockquote>
<h2 id="why-elixir-ships-without-http-client">Why Elixir ships without HTTP client?</h2>
<p>It is obvious: because <strong>Erlang/OTP</strong> has HTTP client included!
And you can call it from any <strong>Elixir</strong> application with easy,
using <strong>Erlang</strong> modules <a href="https://elixir-lang.org/crash-course.html#calling-functions">as atoms</a>.</p>
<p>This client is called <strong>httpc</strong>. It’s simple enough, easy usable
and don’t require modifications of your <strong>mix.exs</strong>.</p>
<p>You can find how to use in <a href="http://erlang.org/doc/man/httpc.html">docs</a>.</p>
<p>I’ll show you small example of <strong>get</strong> HTTP request:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># First, we should start `inets` application.</span>
<span class="c1"># `httpc` is part of it:</span>
<span class="no">Application</span><span class="o">.</span><span class="n">ensure_all_started</span><span class="p">(</span><span class="ss">:inets</span><span class="p">)</span>
<span class="c1"># We should start `ssl` application also,</span>
<span class="c1"># if we want to make secure requests:</span>
<span class="no">Application</span><span class="o">.</span><span class="n">ensure_all_started</span><span class="p">(</span><span class="ss">:ssl</span><span class="p">)</span>
<span class="c1"># Now we can make request:</span>
<span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="p">{{</span><span class="s1">'HTTP/1.1'</span><span class="p">,</span> <span class="m">200</span><span class="p">,</span> <span class="s1">'OK'</span><span class="p">},</span> <span class="n">_headers</span><span class="p">,</span> <span class="n">_body</span><span class="p">}</span> <span class="o">=</span>
<span class="ss">:httpc</span><span class="o">.</span><span class="n">request</span><span class="p">(</span><span class="ss">:get</span><span class="p">,</span> <span class="p">{</span><span class="s1">'http://google.com'</span><span class="p">,</span> <span class="p">[]},</span> <span class="p">[],</span> <span class="p">[])</span>
<span class="c1"># `httpc` will follow redirect from `http` to `https` without</span>
<span class="c1"># additional config.</span>
</code></pre></div></div>
<p>Thus, why people don’t use <code class="highlighter-rouge">httpc</code>? There are several reasons:</p>
<ul>
<li><strong>HTTPC</strong> doesn’t have caching system</li>
<li>It doesn’t use connection pools</li>
<li>It has no streaming out of the box</li>
<li>It doesn’t ship with <strong>MIME</strong> types support</li>
<li>…</li>
<li>Well. Fairly saying, it has nothing in it. It’s dumb simple.</li>
</ul>
<p>But! The question is:</p>
<blockquote>
<p>Do I need all this stuff in my project? Or my requests are <strong>really</strong> simple?</p>
</blockquote>
<p>Let’s for example look at very popular <strong>Hex</strong> package - <a href="https://hex.pm/packages/tzdata"><strong>tzdata</strong></a>. It has only one dependency - <strong>Hackney</strong>.
Let’s investigate a bit, how it’s used.</p>
<p>We can <a href="https://github.com/lau/tzdata/search?utf8=%E2%9C%93&q=%3Ahackney&type=">search</a>
repo by <code class="highlighter-rouge">:hackney</code> query, and find all places where it is used.
Except for <code class="highlighter-rouge">mix.exs</code> and <code class="highlighter-rouge">mix.lock</code>, it’s used in only one place: <strong>Tzdata.DataLoader</strong>.</p>
<p>DataLoader is used to retrieve new timezone data. This data is stored in <strong>tar.gz</strong>
archive on remote server, and is fetched via <code class="highlighter-rouge">:hackney.get</code> method.</p>
<hr />
<p><strong>tzdata/data_loader.ex</strong></p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="n">download_new</span><span class="p">(</span><span class="n">url</span> <span class="p">\\</span> <span class="nv">@download_url</span><span class="p">)</span> <span class="k">do</span>
<span class="no">Logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sd">"</span><span class="s2">Tzdata downloading new data from </span><span class="si">#{</span><span class="n">url</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
<span class="n">set_latest_remote_poll_date</span><span class="p">()</span>
<span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="m">200</span><span class="p">,</span> <span class="n">headers</span><span class="p">,</span> <span class="n">client_ref</span><span class="p">}</span> <span class="o">=</span> <span class="ss">:hackney</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">,</span> <span class="p">[],</span> <span class="sd">"</span><span class="s2">"</span><span class="p">,</span> <span class="p">[</span><span class="ss">follow_redirect:</span> <span class="no">true</span><span class="p">])</span>
<span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="n">body</span><span class="p">}</span> <span class="o">=</span> <span class="ss">:hackney</span><span class="o">.</span><span class="n">body</span><span class="p">(</span><span class="n">client_ref</span><span class="p">)</span>
<span class="n">content_length</span> <span class="o">=</span> <span class="n">byte_size</span><span class="p">(</span><span class="n">body</span><span class="p">)</span>
<span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="n">last_modified</span><span class="p">}</span> <span class="o">=</span> <span class="n">last_modified_from_headers</span><span class="p">(</span><span class="n">headers</span><span class="p">)</span>
<span class="n">new_dir_name</span> <span class="o">=</span>
<span class="sd">"</span><span class="si">#{</span><span class="n">data_dir</span><span class="p">()</span><span class="si">}</span><span class="s2">/tmp_downloads/</span><span class="si">#{</span><span class="n">content_length</span><span class="si">}</span><span class="s2">_</span><span class="si">#{</span><span class="ss">:random</span><span class="o">.</span><span class="n">uniform</span><span class="p">(</span><span class="m">100_000_000</span><span class="p">)</span><span class="si">}</span><span class="s2">/"</span>
<span class="no">File</span><span class="o">.</span><span class="n">mkdir_p!</span><span class="p">(</span><span class="n">new_dir_name</span><span class="p">)</span>
<span class="n">target_filename</span> <span class="o">=</span> <span class="sd">"</span><span class="si">#{</span><span class="n">new_dir_name</span><span class="si">}</span><span class="s2">latest.tar.gz"</span>
<span class="no">File</span><span class="o">.</span><span class="n">write!</span><span class="p">(</span><span class="n">target_filename</span><span class="p">,</span> <span class="n">body</span><span class="p">)</span>
<span class="n">extract</span><span class="p">(</span><span class="n">target_filename</span><span class="p">,</span> <span class="n">new_dir_name</span><span class="p">)</span>
<span class="n">release_version</span> <span class="o">=</span> <span class="n">release_version_for_dir</span><span class="p">(</span><span class="n">new_dir_name</span><span class="p">)</span>
<span class="no">Logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sd">"</span><span class="s2">Tzdata data downloaded. Release version </span><span class="si">#{</span><span class="n">release_version</span><span class="si">}</span><span class="s2">."</span><span class="p">)</span>
<span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="n">content_length</span><span class="p">,</span> <span class="n">release_version</span><span class="p">,</span> <span class="n">new_dir_name</span><span class="p">,</span> <span class="n">last_modified</span><span class="p">}</span>
<span class="k">end</span>
</code></pre></div></div>
<hr />
<p>What do you think, can you fetch one small file in a single day, using
<strong>httpc</strong>? I can!</p>
<p>Fairly saying, I don’t know why <strong>tzdata</strong> maintainers are using <strong>Hackney</strong>
for this task. May be they know something, that I don’t know.
But this is a good example of a place in code, where you can think about what to do - get side package with bench of dependencies, or use Erlang’s <em>batteries</em>, because it’s just <strong>enough</strong>.</p>
<h2 id="httpc-alternatives">HTTPC alternatives</h2>
<p>As we know now, <strong>HTTPC</strong> has one big benefit - it’s included into <strong>OTP</strong>, and a lot of limitation.
So, great developers created there own HTTP clients, and shared them into open source.
In <strong>Hex</strong> registry we can find some good examples of HTTP client alternatives.
Two gigants:</p>
<ul>
<li><a href="https://hex.pm/packages/hackney"><strong>Hackney</strong></a></li>
<li><a href="https://hex.pm/packages/ibrowse"><strong>IBrowse</strong></a></li>
</ul>
<p>and banch of specific clients:</p>
<ul>
<li><a href="https://hex.pm/packages/fusco"><strong>fusco</strong></a></li>
<li><a href="https://hex.pm/packages/gun"><strong>gun</strong></a></li>
<li><a href="https://hex.pm/packages/lhttpc"><strong>lhttpc</strong></a></li>
</ul>
<p>All of them have almost the same user experience and differs in bench of small and very useful <em>features</em>.</p>
<p>We’ll not cover small libraries in this article, and probably in our projects because of simple <strong>open source law</strong>:</p>
<blockquote>
<p>With big popularity comes big quality</p>
</blockquote>
<p>This quality comes from common cookbooks, solutions, documentation and issues on <strong>Github</strong>, and answered questions on <strong>StackOverflow</strong>.</p>
<p>That’s why, lets try to compare a bit our gigant!</p>
<h3 id="performance">Performance</h3>
<p>The main thing, that is important for developer - library performance. Everybody loves benchmarks (even if they don’t know what to do with them)!</p>
<p>I’ve found interesting repo - <a href="https://github.com/ransomr/httpcbench">httpcbench</a>,
which has some information about performance comparison of main HTTP clients.</p>
<p>I’ll bring the <strong>Results</strong> table here:</p>
<hr />
<table>
<thead>
<tr>
<th>Client</th>
<th style="text-align: right">runtime</th>
<th style="text-align: right">wall_clock</th>
<th style="text-align: right">mem</th>
<th style="text-align: right">failures</th>
</tr>
</thead>
<tbody>
<tr>
<td>hackney (default pool)</td>
<td style="text-align: right">38560</td>
<td style="text-align: right">30912</td>
<td style="text-align: right">16.110</td>
<td style="text-align: right">0</td>
</tr>
<tr>
<td>httpc</td>
<td style="text-align: right">34080</td>
<td style="text-align: right">27913</td>
<td style="text-align: right">54.083</td>
<td style="text-align: right">0</td>
</tr>
<tr>
<td>httpc (optimized)</td>
<td style="text-align: right">33540</td>
<td style="text-align: right">26981</td>
<td style="text-align: right">55.402</td>
<td style="text-align: right">0</td>
</tr>
<tr>
<td>ibrowse</td>
<td style="text-align: right">212720</td>
<td style="text-align: right">112853</td>
<td style="text-align: right">14.567</td>
<td style="text-align: right">0</td>
</tr>
<tr>
<td>ibrowse (optimized)</td>
<td style="text-align: right">22410</td>
<td style="text-align: right">21029</td>
<td style="text-align: right">59.849</td>
<td style="text-align: right">0</td>
</tr>
<tr>
<td>lhttpc</td>
<td style="text-align: right">27820</td>
<td style="text-align: right">29276</td>
<td style="text-align: right">12.893</td>
<td style="text-align: right">0</td>
</tr>
</tbody>
</table>
<hr />
<p>Unfortunately, the repo’s last commit was mad 3 years ago. But it’s not a problem for us!
We can fix these test with all modern versions of these libraries.</p>
<p>You can find rewritten into <strong>Elixir</strong> code <a href="https://github.com/Virviil/ex_http_bench">in this repo</a>.</p>
<p>I’ll bring this table also:</p>
<hr />
<table>
<thead>
<tr>
<th>Client</th>
<th>Runtime</th>
<th>Wall clock</th>
<th>Memory (MB)</th>
<th>Failures</th>
</tr>
</thead>
<tbody>
<tr>
<td>Hackney (default pool)</td>
<td>59691</td>
<td>50888</td>
<td>14.837</td>
<td>0</td>
</tr>
<tr>
<td>HTTPC</td>
<td>47906</td>
<td>42531</td>
<td>27.059</td>
<td>0</td>
</tr>
<tr>
<td>HTTPC Optimized</td>
<td>50802</td>
<td>45300</td>
<td>28.631</td>
<td>0</td>
</tr>
<tr>
<td>LHTTPC</td>
<td>46361</td>
<td>38610</td>
<td>80.735</td>
<td>0</td>
</tr>
<tr>
<td>IBrowse</td>
<td>58724</td>
<td>49948</td>
<td>15.141</td>
<td>0</td>
</tr>
<tr>
<td>IBrowse (optimized)</td>
<td>58199</td>
<td>49791</td>
<td>11.781</td>
<td>0</td>
</tr>
</tbody>
</table>
<hr />
<p>Of course all benchmarks are synthetic. But they show, that there is almost no difference between our libraries.</p>
<p><strong>Conclusion</strong>: they both performs almost the same. And we move next.</p>
<h3 id="security">Security</h3>
<p>The second (and sometimes - the first) parameter - security. We should be sure, that our requests goes to right domains, can’t be penetrated, nobody can steal our tokens and keys.</p>
<p>Here comes SSL comparison.</p>
<p>There is <a href="https://blog.voltone.net/post/7">great article about security</a>, and no sense to say something more.</p>
<p><strong>Conclusion</strong>: Only <strong>Hackney</strong> validates <strong>SSL</strong> by dafault, using <strong>certifi</strong> library, but you can use this library in every other HTTP client. But:</p>
<ul>
<li>you can forget to do this</li>
<li>you should create additional amount of code by yourself to implement this.</li>
</ul>
<p>So, from security point of view, <strong>Hackney</strong> looks the most considered.</p>
<h3 id="secondary-parameters">Secondary parameters</h3>
<p>Fairly saying, <strong>Hackney</strong> has some <em>secondary</em> advantages:</p>
<ul>
<li>3.5M on <strong>Hex</strong> versus <strong>IBrowse</strong>’s 0.5M</li>
<li>Binary strings with Unicode support, which is more common in <strong>Elixir</strong>, against charlists, that are more common in <strong>Erlang</strong> world</li>
<li><strong>Hackney</strong>’s last release was 1 day ago versus <strong>IBrowse</strong>’s previous year release (don’t blame me for falsifications with this article’s posting time :) )</li>
</ul>
<h3 id="results">Results</h3>
<p>Both <strong>IBrowse</strong> and <strong>Hackeny</strong> are great instruments to make good HTTP communication for your application. But, as for me, I’ll choose <strong>Hackney</strong> for these reason:</p>
<ul>
<li>I have 5 time bigger chance to have it already installed in my project’s deps.</li>
<li>I don’t need to think about HTTP client’s security</li>
<li>…</li>
</ul>
<p>No more reasons. Really… But these two should be enough!</p>
<h2 id="what-about-pure-elixir-implementations">What about pure Elixir implementations?</h2>
<p>I don’t want to disappoint you, but there is no famous <strong>Elixir</strong> HTTP client.
All <strong>Elixir</strong>’s <em>HTTP client</em> packages are just wrappers for <strong>Erlang</strong> written
libraries. For example</p>
<ul>
<li><strong>HTTPoison</strong> - is based on <strong>Hackney</strong>, and inspired by…</li>
<li><strong>HTTPotion</strong>, which is based on <strong>IBrowse</strong></li>
<li><strong>Tesla</strong> - has adapters for both <strong>Hackney</strong> and <strong>IBrowse</strong>, and even to <strong>httpc</strong></li>
<li><strong>simplehttp</strong> (you’ve never heard about it, yeah?) - based only on <strong>httpc</strong></li>
</ul>
<p>After getting this information, one should ask himself:</p>
<blockquote>
<p>Why should I wrap <strong>Erlang</strong> library into <strong>Elixir</strong> library, when I can call it directly?</p>
</blockquote>
<p>This question have several possible answers:</p>
<ul>
<li>Wrapper makes the using experience smoother</li>
<li>Wrapper brings additional functionality</li>
</ul>
<p>Let’s observe these reasons one by one.</p>
<h3 id="smoother-experience">Smoother experience</h3>
<p>When one says:</p>
<blockquote>
<p>This library makes my coding experience smoother</p>
</blockquote>
<p>in terms of <strong>Erlang</strong> and <strong>Elixir</strong>, the meaning of this statement is:</p>
<ol>
<li><strong><em>Native</em> calls.</strong> In Elixir, we have common way to call functions, that looks something like:
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="no">MyNamespace</span><span class="o">.</span><span class="no">MyModule</span><span class="o">.</span><span class="n">my_function</span><span class="p">(</span><span class="n">arg_1</span><span class="p">,</span> <span class="o">...</span><span class="p">,</span> <span class="n">arg_n</span><span class="p">)</span>
</code></pre></div> </div>
<p>As you see, <strong>MyModule</strong> is <strong>CamelCased</strong> word, that can be prefixed with <strong>MyNamespace</strong> (also <strong>CammelCased</strong>), followed by function name with function arguments in brackets.</p>
<p>In comparison, <strong>Erlang</strong> functions will be called somthing like this:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="ss">:erlang_module</span><span class="o">.</span><span class="n">my_function</span><span class="p">(</span><span class="n">arg_1</span><span class="p">,</span> <span class="o">...</span><span class="p">,</span> <span class="n">arg_n</span><span class="p">)</span>
</code></pre></div> </div>
<p><strong>Elixir</strong> modules can be aliased, imported and required:</p>
<ul>
<li><strong>alias</strong> - simplifies module calles by removing namespace prefixes</li>
<li><strong>import</strong> - imports module’s functions in current scope</li>
<li><strong>require</strong> - brings <em>macroses</em> from module into current scope</li>
</ul>
<p>You can also import <strong>Erlang</strong> modules</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="n">iex</span><span class="o">></span> <span class="kn">import</span> <span class="ss">:erlang</span>
<span class="n">iex</span><span class="o">></span> <span class="n">time</span><span class="p">()</span> <span class="c1"># here goes :erlang.time() call</span>
</code></pre></div> </div>
<p>while <em>requiring</em> and <em>aliasing</em> has no sense here.</p>
<p><strong>Erlang</strong> calls even have autocomplition in <strong>IEx</strong>, the same as <strong>Elixir</strong> functions</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="n">iex</span><span class="o">></span> <span class="ss">:timer</span><span class="o">.</span><span class="n">sl</span><span class="p">[</span><span class="no">Tab</span><span class="p">]</span>
<span class="c1"># goes to</span>
<span class="n">iex</span><span class="o">></span> <span class="ss">:timer</span><span class="o">.</span><span class="n">sleep</span>
</code></pre></div> </div>
<p><strong>Conclusion</strong>: There is <strong>almost no</strong> difference between these calls, so the experience should be almost the same.</p>
</li>
<li>
<p><strong>Documentation</strong>. During development process, you can get documentation for your <strong>Elixir</strong> functions in two common different ways:</p>
<ul>
<li>Using <strong>IEx</strong>, one can call <code class="highlighter-rouge">h/1</code> function to get documentation on given function:
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="n">iex</span><span class="o">></span> <span class="n">h</span> <span class="no">Enum</span><span class="o">.</span><span class="n">at</span>
<span class="n">iex</span><span class="p">(</span><span class="m">7</span><span class="p">)</span><span class="o">></span> <span class="n">h</span> <span class="no">Enum</span><span class="o">.</span><span class="n">at</span>
<span class="k">def</span> <span class="n">at</span><span class="p">(</span><span class="n">enumerable</span><span class="p">,</span> <span class="n">index</span><span class="p">,</span> <span class="n">default</span> <span class="p">\\</span> <span class="no">nil</span><span class="p">)</span>
<span class="nv">@spec</span> <span class="n">at</span><span class="p">(</span><span class="n">t</span><span class="p">(),</span> <span class="n">index</span><span class="p">(),</span> <span class="n">default</span><span class="p">())</span> <span class="p">::</span> <span class="n">element</span><span class="p">()</span> <span class="o">|</span> <span class="n">default</span><span class="p">()</span>
<span class="no">Finds</span> <span class="n">the</span> <span class="n">element</span> <span class="n">at</span> <span class="n">the</span> <span class="n">given</span> <span class="n">index</span> <span class="p">(</span><span class="n">zero</span><span class="o">-</span><span class="n">based</span><span class="p">)</span><span class="o">.</span>
<span class="o">...</span>
</code></pre></div> </div>
</li>
<li>This functionality can be included into you <strong>IDE</strong>.</li>
</ul>
<p><strong>None</strong> of this works for <strong>Erlang</strong> functions for the moment. You can be lucky and get <strong>spec</strong>,
but also not for every function:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="n">iex</span><span class="o">></span> <span class="n">h</span> <span class="ss">:timer</span><span class="o">.</span><span class="n">sleep</span>
<span class="ss">:timer</span><span class="o">.</span><span class="n">sleep</span><span class="o">/</span><span class="m">1</span>
<span class="nv">@spec</span> <span class="n">sleep</span><span class="p">(</span><span class="n">time</span><span class="p">)</span> <span class="p">::</span> <span class="ss">:ok</span> <span class="ow">when</span> <span class="ss">Time:</span> <span class="n">timeout</span><span class="p">(),</span> <span class="ss">time:</span> <span class="n">var</span>
<span class="no">Documentation</span> <span class="n">is</span> <span class="ow">not</span> <span class="n">available</span> <span class="n">for</span> <span class="n">non</span><span class="o">-</span><span class="no">Elixir</span> <span class="n">modules</span><span class="o">.</span> <span class="no">Showing</span> <span class="n">only</span> <span class="n">specs</span><span class="o">.</span>
</code></pre></div> </div>
<p>This can be a big problem, when you shouldn’t go googling with <strong>Elixir</strong> library, but need to do this for <strong>Erlang</strong> one. But this seems not to be working for wrappers.</p>
<p>For example, this is the line in <strong>HTTPosion</strong> documentations:</p>
<blockquote>
<p>:ssl - SSL options supported by the ssl erlang module</p>
</blockquote>
<p>Well, this mean that developer should go into <strong>Erlang</strong>’s <code class="highlighter-rouge">ssl</code> module documentation to understand what the parameters should be used in <strong>Elixir</strong>’s <strong>HTTPoison</strong> library.</p>
<p>And both <strong>HTTPoison</strong> and <strong>HTTPotion</strong> lay on docs to <strong>Hackney</strong> and <strong>IBrowse</strong>, thus to make something more complicated then simple <code class="highlighter-rouge">get</code> you should read there docks also.</p>
<p><strong>Conclusion</strong>: in both versions you probably will go into google or docks via Internet, so the experience should also be <strong>almost the same</strong>.</p>
</li>
</ol>
<h3 id="additional-functionality">Additional functionality</h3>
<p>Our <strong>Elixir</strong> wrappers comes with some portion of additional functionality:</p>
<ul>
<li><strong>HTTPoison</strong> and <strong>HTTPotion</strong> wraps <em>errors</em> into <strong>Elixir</strong> structs.</li>
<li>They both also bring some <em>metaprogramming magic</em> to build <strong>API wrappers</strong></li>
<li><strong>Tornado</strong> entirely changes the way of building request and wrappers.</li>
</ul>
<p>The question is the same again:</p>
<blockquote>
<p>Do I need all this stuff in my project?</p>
</blockquote>
<p>Let’s also make some investigations upon bench of libraries from <strong>Hex</strong>. We will do the same process, as with <strong>tzdata</strong>:</p>
<ul>
<li>Search for <strong>HTTPoison</strong> calls in repo.</li>
<li>Analyze, can it be simplified by pure <strong>Erlang</strong>’s <strong>Hackney</strong> calls</li>
</ul>
<ol>
<li><a href="https://hex.pm/packages/recaptcha"><strong>Recaptacha</strong></a> - performs communication with <strong>recaptcha</strong>’s servers to make captcha validation.
<ul>
<li>Search results can be find <a href="https://github.com/samueljseay/recaptcha/search?utf8=%E2%9C%93&q=HTTPoison&type=">here</a>. As we see, there is only <strong>one</strong> HTTP request (<strong>POST</strong>).</li>
<li>We can change <code class="highlighter-rouge">HTTPoison</code> into <code class="highlighter-rouge">:hackney</code> just in code with almost zero remaking. <strong>HTTPoiso.Base</strong> is not used here, <strong>HTTPoison.Error.t()</strong> - also. May be we even can use <strong>:httpc</strong> here, what do you think about that?</li>
</ul>
</li>
<li><a href=""><strong>random_user_api</strong></a> - communicates with server to get <em>random user</em>.
<ul>
<li>Search results can be find <a href="https://github.com/PatNowak/random_user_api/search?utf8=%E2%9C%93&q=HTTPoison&type=">here</a>. As we see, there is only <strong>one</strong> HTTP request (<strong>GET</strong>).</li>
<li>The same as previous library, nothing to add.</li>
</ul>
</li>
<li><a href=""><strong>pubnux</strong></a> - communicates with <strong>PubNub</strong> server.
<ul>
<li>Search results can be find <a href="https://github.com/Liftitapp/pubnux/search?utf8=%E2%9C%93&q=HTTPoison&type=">here</a>. It’s really hard to find library with <strong>HTTPoison.Base</strong>, as you see :)</li>
<li>The same as previous, no sense to continue here anymore.</li>
</ul>
</li>
<li>And so on, and so on… You can investigate them by yourself using for example <a href="https://hex.pm/packages?search=depends%3Ahttpoison">this list</a> (yes, <strong>HTTPoison</strong> dependents).</li>
</ol>
<h2 id="conclusion">Conclusion</h2>
<p>People are very lazy creatures. And software developers are the laziest people :)</p>
<p>It’s very easy to bring <strong>ten</strong> dependencies into your project just to make single HTTP request.
<strong>Harder</strong> - to deal with possible <strong>depndency hell</strong>, when your project grows.
<strong>Harder</strong> - to track and support in <strong>OSS</strong> <strong>N+1</strong> library then <strong>N</strong>…</p>
<p>I think, that after reading this article, you will think a bit before planning your <strong>API</strong> communication. And this will simplify your development experience bigger, that <em>no use of erlang ibraries in elixir</em>.</p>
<p>And if you need concrete solutions, here they are:</p>
<ol>
<li>If you need to make single call in your small pet project, use <strong>:httpc</strong>. It’s <em>enough</em> to do the task, while not bringing a lot of deps in code.</li>
<li>If you develop your own <strong>API wrapper</strong>, either for your project use, or for publishing in <strong>Hex</strong>, <strong>Hackney</strong> should be just <em>enough</em> to do this. Moving from <strong>Elixir</strong> wrappers to pure library will reduce your deps folder, and will help to avoid <em>adapter bugs</em>.</li>
</ol>
<h2 id="acknowledgments">Acknowledgments</h2>
<p>Special thanks to <strong>maintainers</strong> and <strong>contributors</strong> of dissected libraries. They are doing really great job and bring our <strong>Elixir</strong> community higher.</p>The process of making HTTP requests in Elixir seems to be obvious for every developer: one should take HTTPoison (3,5M downloads from Hex.pm), and do whatever he wants! But… Have you ever thought about alternatives? If not - follow this article, and get the answer to the question: How to make HTTP request?Elixir: Brackets Hell in Supervision tree2017-10-25T00:00:00+00:002017-10-25T00:00:00+00:00https://virviil.github.io/2017/10/25/elixir-brackets-hell-in-supervision-tree<p><img src="https://4everinbeta.files.wordpress.com/2015/01/supervisiontree.png" alt="Supervision tree" /></p>
<p>The <code class="highlighter-rouge">Supervisor.Spec</code> module was deprecated and does not work with the module-based child specs,
introduced in <strong>Elixir v1.5</strong>. Thus, all methods for supervision tree declaring were sugnificantly changed.
It’s great time to understand the <em>waterflow</em> of passing arguments -
from top-level <code class="highlighter-rouge">Supervisor</code> to low-level workers aka <code class="highlighter-rouge">GenServers</code>.
<!--more--></p>
<h2 id="the-problem">The problem</h2>
<p>I faced the problem with declaring arguments in <strong>Supervisor -> GenServer</strong> chain.
For example, look at the <strong>Elixir’s Supervisor</strong>
<a href="https://hexdocs.pm/elixir/Supervisor.html#module-child-specification">docks</a>. You can find this code there:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">Stack</span><span class="o">.</span><span class="n">child_spec</span><span class="p">([</span><span class="ss">:hello</span><span class="p">])</span>
<span class="c1">#=> %{</span>
<span class="ss">id:</span> <span class="no">Stack</span><span class="p">,</span>
<span class="ss">start:</span> <span class="p">{</span><span class="no">Stack</span><span class="p">,</span> <span class="ss">:start_link</span><span class="p">,</span> <span class="p">[[</span><span class="ss">:hello</span><span class="p">]]},</span>
<span class="ss">restart:</span> <span class="ss">:permanent</span><span class="p">,</span>
<span class="ss">shutdown:</span> <span class="m">5000</span><span class="p">,</span>
<span class="ss">type:</span> <span class="ss">:worker</span>
<span class="p">}</span>
</code></pre></div></div>
<p>What the hack is going on here <strong>o_O</strong>!? Why <code class="highlighter-rouge">:hello</code> is wrapped into <em>single brackets</em> in the first line,
but in <em>double brackets</em> in the forth line?</p>
<p>I’m absolutely not sure how to pass, for example, <code class="highlighter-rouge">:ok</code> into <code class="highlighter-rouge">GenServer</code> - using <code class="highlighter-rouge">:ok</code>,
or maybe <code class="highlighter-rouge">[:ok]</code>, or maybe <code class="highlighter-rouge">[[:ok]]</code>…</p>
<p>How to match this value in <code class="highlighter-rouge">start_link</code>, <code class="highlighter-rouge">chils_spec</code> and <code class="highlighter-rouge">init</code> functions?..</p>
<p>If you feel the same - let’s dive into this <em>brackets hell</em> together.</p>
<h2 id="init1-has-arity-1-one">init/1 has arity 1 (<em>ONE</em>).</h2>
<p>I hope that the good idea is to start from the ground - from the function,
when you <strong>really</strong> understand what to do with this arguments.
The only function with such parameters is function <code class="highlighter-rouge">init/1</code> in <code class="highlighter-rouge">GenServer</code>.</p>
<p>For sure, you are initializing your worker’s state with passed arguments:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="n">init</span><span class="p">(</span><span class="n">_something_</span><span class="p">)</span> <span class="k">do</span>
<span class="n">state</span> <span class="o">=</span> <span class="o">...</span>
<span class="c1"># some arguments conversions</span>
<span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="n">state</span><span class="p">}</span>
<span class="k">end</span>
</code></pre></div></div>
<p>The main idea here, and the only thing you have to learn by heart:</p>
<blockquote>
<p>init/1 has arity 1!</p>
</blockquote>
<p>If you need more than one argument - you have to wrap them into some <strong>data structure</strong>,
so it can be passed into function with arity <em>1</em>.</p>
<p>Here comes two main errors:</p>
<ul>
<li>
<p>You have only one argument (for example, you done have arguments at all, or you want to pass <code class="highlighter-rouge">true</code> or <code class="highlighter-rouge">:ok</code>)</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="n">init</span><span class="p">([</span><span class="no">true</span><span class="p">])</span> <span class="k">do</span>
<span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="p">%{}}</span>
<span class="k">end</span>
</code></pre></div> </div>
<p>Don’t you think, that the brackets here are unnecessary? I’m sure, that they do!
Brackets here have no sense, so - just don’t use them!</p>
</li>
<li>
<p>You have arguments, that already wraped into <code class="highlighter-rouge">tuple</code>, <code class="highlighter-rouge">map</code> or <code class="highlighter-rouge">list</code>.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">map</span> <span class="o">=</span> <span class="p">%{</span><span class="ss">name:</span> <span class="sd">"</span><span class="s2">Joe"</span><span class="p">,</span> <span class="ss">age:</span> <span class="m">21</span><span class="p">}</span>
<span class="n">tuple</span> <span class="o">=</span> <span class="p">{</span><span class="sd">"</span><span class="s2">Joe"</span><span class="p">,</span> <span class="m">21</span><span class="p">}</span>
<span class="n">list</span> <span class="o">=</span> <span class="p">[</span><span class="sd">"</span><span class="s2">Joe"</span><span class="p">,</span> <span class="m">21</span><span class="p">]</span>
</code></pre></div> </div>
<p>And you are trying to wrap them into brackets?</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="n">init</span><span class="p">([%{</span><span class="ss">name:</span> <span class="sd">"</span><span class="s2">Joe"</span><span class="p">,</span> <span class="ss">age:</span> <span class="m">21</span><span class="p">}])</span> <span class="k">do</span>
<span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="p">%{</span><span class="ss">name:</span> <span class="sd">"</span><span class="s2">Joe"</span><span class="p">,</span> <span class="ss">age:</span> <span class="m">21</span><span class="p">}}</span>
<span class="k">end</span>
<span class="k">def</span> <span class="n">init</span><span class="p">([{</span><span class="sd">"</span><span class="s2">Joe"</span><span class="p">,</span> <span class="m">21</span><span class="p">}])</span> <span class="k">do</span>
<span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="p">{</span><span class="sd">"</span><span class="s2">Joe"</span><span class="p">,</span> <span class="m">21</span><span class="p">}}</span>
<span class="k">end</span>
<span class="k">def</span> <span class="n">init</span><span class="p">([[</span><span class="sd">"</span><span class="s2">Joe"</span><span class="p">,</span> <span class="m">21</span><span class="p">]])</span> <span class="k">do</span>
<span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="p">[</span><span class="sd">"</span><span class="s2">Joe"</span><span class="p">,</span> <span class="m">21</span><span class="p">]}</span>
<span class="k">end</span>
</code></pre></div> </div>
<p>Pretty nice picture… With tones of unnecessary brackets!</p>
</li>
</ul>
<p>One more thing to say: <code class="highlighter-rouge">use GenServer</code> brings us <code class="highlighter-rouge">init\1</code> function, defined like this:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="n">init</span><span class="p">(</span><span class="n">state</span><span class="p">)</span> <span class="k">do</span>
<span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="n">state</span><span class="p">}</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Basically saying, it’s the only one reason to add <strong>one</strong> argument in <strong>brackets</strong>:</p>
<ul>
<li>you <strong>want</strong> your <strong>GenServer’s state</strong> to be <strong>list</strong> with <strong>one</strong> element after initialization</li>
<li>you <strong>don’t want</strong> to <strong>redefine</strong> <code class="highlighter-rouge">init/1</code></li>
</ul>
<p>And to conclude:</p>
<ul>
<li>when you see <code class="highlighter-rouge">GenServer</code> that doesn’t have <code class="highlighter-rouge">init/1</code> defined in the code - you know how this <code class="highlighter-rouge">init</code> is done.</li>
<li>if your <code class="highlighter-rouge">init/1</code> does the same as predefined <code class="highlighter-rouge">init/1</code> - don’t override it manualy in the code!</li>
</ul>
<p>Don’t use brackets just because you’ve seen brackets in the documentation - probably,
the example there is not so good. Try to think your own head - and continue reading.</p>
<h2 id="genserverstart_link---you-already-know-what-to-do">GenServer.start_link - you already know what to do!</h2>
<p>If one want’s to pass the data into <code class="highlighter-rouge">init/1</code> function of your GenServer module -
he should start the server via <code class="highlighter-rouge">GenServer.start_link/3</code> function. Let’s look at it’s spec:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">start_link</span><span class="p">(</span><span class="ss">module:</span> <span class="n">atom</span><span class="p">,</span> <span class="ss">args:</span> <span class="n">any</span><span class="p">,</span> <span class="ss">options:</span> <span class="no">Keyword</span><span class="o">.</span><span class="n">t</span><span class="p">)</span> <span class="p">::</span> <span class="n">on_start</span>
</code></pre></div></div>
<p>Well, here everything is easy - you have only <em>one</em> place for args,
and this args will pass into <code class="highlighter-rouge">init/1</code> function with arity <strong>one</strong>.
You should learn nothing - you already know what to do!</p>
<p>Let’s look at some examples:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># Good</span>
<span class="no">GenServer</span><span class="o">.</span><span class="n">start_link</span><span class="p">(</span><span class="bp">__MODULE__</span><span class="p">,</span> <span class="no">true</span><span class="p">)</span>
<span class="k">def</span> <span class="n">init</span><span class="p">(</span><span class="no">true</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="o">...</span>
<span class="no">GenServer</span><span class="o">.</span><span class="n">start_link</span><span class="p">(</span><span class="bp">__MODULE__</span><span class="p">,</span> <span class="p">%{</span><span class="ss">name:</span> <span class="sd">"</span><span class="s2">Joe"</span><span class="p">,</span> <span class="ss">age:</span> <span class="m">21</span><span class="p">})</span>
<span class="k">def</span> <span class="n">init</span><span class="p">(%{</span><span class="ss">name:</span> <span class="sd">"</span><span class="s2">Joe"</span><span class="p">,</span> <span class="ss">age:</span> <span class="m">21</span><span class="p">}),</span> <span class="k">do</span><span class="p">:</span> <span class="o">...</span>
<span class="no">GenServer</span><span class="o">.</span><span class="n">start_link</span><span class="p">(</span><span class="bp">__MODULE__</span><span class="p">,</span> <span class="p">{</span><span class="sd">"</span><span class="s2">Joe"</span><span class="p">,</span> <span class="m">21</span><span class="p">})</span>
<span class="k">def</span> <span class="n">init</span><span class="p">({</span><span class="sd">"</span><span class="s2">Joe"</span><span class="p">,</span> <span class="m">21</span><span class="p">}),</span> <span class="k">do</span><span class="p">:</span> <span class="o">...</span>
<span class="no">GenServer</span><span class="o">.</span><span class="n">start_link</span><span class="p">(</span><span class="bp">__MODULE__</span><span class="p">,</span> <span class="p">[</span><span class="sd">"</span><span class="s2">Joe"</span><span class="p">,</span> <span class="m">21</span><span class="p">])</span>
<span class="k">def</span> <span class="n">init</span><span class="p">([</span><span class="sd">"</span><span class="s2">Joe"</span><span class="p">,</span> <span class="m">21</span><span class="p">]),</span> <span class="k">do</span><span class="p">:</span> <span class="o">...</span>
<span class="c1"># Never Ever Do This at Home</span>
<span class="no">GenServer</span><span class="o">.</span><span class="n">start_link</span><span class="p">(</span><span class="bp">__MODULE__</span><span class="p">,</span> <span class="p">[</span><span class="no">true</span><span class="p">])</span>
<span class="k">def</span> <span class="n">init</span><span class="p">(</span><span class="no">true</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="o">...</span>
<span class="o">**</span> <span class="p">(</span><span class="no">FunctionClauseError</span><span class="p">)</span> <span class="n">no</span> <span class="n">function</span> <span class="n">clause</span> <span class="n">matching</span>
<span class="no">GenServer</span><span class="o">.</span><span class="n">start_link</span><span class="p">(</span><span class="bp">__MODULE__</span><span class="p">,</span> <span class="p">[</span><span class="no">true</span><span class="p">])</span>
<span class="k">def</span> <span class="n">init</span><span class="p">([</span><span class="no">true</span><span class="p">]),</span> <span class="k">do</span><span class="p">:</span> <span class="o">...</span>
<span class="c1"># 4 unnecessary brackets...</span>
<span class="no">GenServer</span><span class="o">.</span><span class="n">start_link</span><span class="p">(</span><span class="bp">__MODULE__</span><span class="p">,</span> <span class="p">[{</span><span class="sd">"</span><span class="s2">Joe"</span><span class="p">,</span> <span class="m">21</span><span class="p">}])</span>
<span class="k">def</span> <span class="n">init</span><span class="p">({</span><span class="sd">"</span><span class="s2">Joe"</span><span class="p">,</span> <span class="m">21</span><span class="p">}),</span> <span class="k">do</span><span class="p">:</span> <span class="o">...</span>
<span class="o">**</span> <span class="p">(</span><span class="no">FunctionClauseError</span><span class="p">)</span> <span class="n">no</span> <span class="n">function</span> <span class="n">clause</span> <span class="n">matching</span>
<span class="no">GenServer</span><span class="o">.</span><span class="n">start_link</span><span class="p">(</span><span class="bp">__MODULE__</span><span class="p">,</span> <span class="p">[[</span><span class="sd">"</span><span class="s2">Joe"</span><span class="p">,</span> <span class="m">21</span><span class="p">]])</span>
<span class="k">def</span> <span class="n">init</span><span class="p">([[</span><span class="sd">"</span><span class="s2">Joe"</span><span class="p">,</span> <span class="m">21</span><span class="p">]]),</span> <span class="k">do</span><span class="p">:</span> <span class="o">...</span>
<span class="c1"># C'mon, are you kidding me?</span>
</code></pre></div></div>
<p>Yep, as you see - it’s very easy. But the hard part is coming…</p>
<h2 id="__module__start_link---what-to-do-here">__MODULE__.start_link - what to do here?</h2>
<p>The obvious place to call <code class="highlighter-rouge">GenServer.start_link/3</code> in your <strong>GenServer</strong>’s code - define your own <code class="highlighter-rouge">start_link</code> function.</p>
<p>First, lets look at the functions that come with <code class="highlighter-rouge">use GenServer</code> macro calling into our <strong>GenServer</strong>:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">iex</span><span class="o">></span> <span class="k">defmodule</span> <span class="no">Foo</span><span class="p">,</span> <span class="k">do</span><span class="p">:</span> <span class="kn">use</span> <span class="no">GenServer</span>
<span class="n">iex</span><span class="o">></span> <span class="no">Foo</span><span class="o">.</span><span class="n">__info__</span><span class="p">(</span><span class="ss">:functions</span><span class="p">)</span>
<span class="p">[</span><span class="ss">child_spec:</span> <span class="m">1</span><span class="p">,</span> <span class="ss">code_change:</span> <span class="m">3</span><span class="p">,</span> <span class="ss">handle_call:</span> <span class="m">3</span><span class="p">,</span> <span class="ss">handle_cast:</span> <span class="m">2</span><span class="p">,</span> <span class="ss">handle_info:</span> <span class="m">2</span><span class="p">,</span> <span class="ss">init:</span> <span class="m">1</span><span class="p">,</span> <span class="ss">terminate:</span> <span class="m">2</span><span class="p">]</span>
</code></pre></div></div>
<p>As we see, <code class="highlighter-rouge">use GenServer</code> have brought many functions to the module,
but <code class="highlighter-rouge">start_link</code> is not in the list. So, we have to define it by ourselves.</p>
<p>Oh, yes! Finally we got a place to play with arity. We can define <code class="highlighter-rouge">start_link</code> with as many arguments as we want!</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># Just example - kick off developer that will do this in real code</span>
<span class="k">def</span> <span class="n">start_link</span><span class="p">(</span><span class="n">_a</span><span class="p">,</span><span class="n">_b</span><span class="p">,</span><span class="n">_c</span><span class="p">,</span> <span class="o">...</span> <span class="n">_z</span><span class="p">)</span> <span class="k">do</span> <span class="c1"># Up to 255 - maximum Erlang arity</span>
<span class="no">GenServer</span><span class="o">.</span><span class="n">start_link</span><span class="p">(</span><span class="bp">__MODULE__</span><span class="p">,</span> <span class="no">true</span><span class="p">)</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Then we can call this <code class="highlighter-rouge">start_link/255</code> function from our code:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="n">pid</span><span class="p">}</span> <span class="o">=</span> <span class="no">MyGenServer</span><span class="o">.</span><span class="n">start_link</span><span class="p">(</span><span class="m">1</span><span class="p">,</span><span class="m">2</span><span class="p">,</span><span class="m">3</span><span class="p">,</span><span class="m">4</span><span class="p">,</span><span class="o">...</span><span class="m">255</span><span class="p">)</span>
</code></pre></div></div>
<p>Well… Why you can need to define <code class="highlighter-rouge">start_link</code> with arity more then <strong>1</strong> in real life?
We’ll not discuss neither “Do I really need this?”, nor “Is this pattern or antipattern?”, just bench of examples:</p>
<ul>
<li>
<p>you want to define dynamic name:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="n">start_link</span><span class="p">(</span><span class="n">init_arg</span><span class="p">,</span> <span class="n">name</span><span class="p">)</span> <span class="k">do</span>
<span class="no">GenServer</span><span class="o">.</span><span class="n">start_link</span><span class="p">(</span><span class="bp">__MODULE__</span><span class="p">,</span> <span class="n">init_arg</span><span class="p">,</span> <span class="ss">name:</span> <span class="n">name</span><span class="p">)</span>
<span class="k">end</span>
</code></pre></div> </div>
</li>
<li>
<p>manipulate with <code class="highlighter-rouge">module</code>, that defines callbacks for your <strong>GenServer</strong>:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="n">start_link</span><span class="p">(</span><span class="n">dynamic_module_name</span><span class="p">,</span> <span class="n">init_arg</span><span class="p">)</span> <span class="k">do</span>
<span class="no">GenServer</span><span class="o">.</span><span class="n">start_link</span><span class="p">(</span><span class="n">dynamic_module_name</span><span class="p">,</span> <span class="n">init_arg</span><span class="p">)</span>
<span class="k">end</span>
</code></pre></div> </div>
</li>
<li>
<p>do something else with opts:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="n">start_link</span><span class="p">(</span><span class="n">init_arg</span><span class="p">,</span> <span class="n">opts</span><span class="p">)</span> <span class="k">do</span>
<span class="no">GenServer</span><span class="o">.</span><span class="n">start_link</span><span class="p">(</span><span class="bp">__MODULE__</span><span class="p">,</span> <span class="n">init_arg</span><span class="p">,</span> <span class="n">opts</span><span class="p">)</span>
<span class="k">end</span>
</code></pre></div> </div>
</li>
</ul>
<p>As you see, nobody knows what arity will be your <code class="highlighter-rouge">start_link</code> function.
And this is a place where problems are beginning. This is a portal to the <strong>brackets hell</strong>…</p>
<h2 id="child-spec-its-dangerous-to-go-alone-take-this">Child spec: It’s dangerous to go alone! Take this</h2>
<p>Before passing through the hellgate - we should arm ourselves with child specification knowledge.
Fairly saying, child spec did not appear in <strong>Elixir 1.5</strong> for the first time.
But you really had no instruments to deal with it in Elixir -
until you didn’t want to dig in some crazy metaprogramming or equal.</p>
<p><strong>Child specification</strong> is used by your <strong>Supervisor</strong> to understand:</p>
<ul>
<li><strong>how</strong> to <strong>start</strong> his children</li>
<li>and <strong>how</strong> to <strong>restart</strong> his children</li>
</ul>
<p>In return, <em>*Supervisor</em>8 has his own specification, and it configures:</p>
<ul>
<li><strong>when</strong> to restart children (<strong>Restart strategy</strong>)</li>
<li>and <strong>restart frequency until suicide</strong></li>
</ul>
<p>Let’s look at the child spec example for previously defined crazy <strong>Foo</strong> module with <code class="highlighter-rouge">start_link/255</code>:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">%{</span>
<span class="ss">id:</span> <span class="no">Foo</span><span class="p">,</span>
<span class="ss">start:</span> <span class="p">{</span><span class="no">Foo</span><span class="p">,</span> <span class="ss">:start_link</span><span class="p">,</span> <span class="p">[</span><span class="m">1</span><span class="p">,</span> <span class="m">2</span><span class="p">,</span> <span class="m">3</span><span class="p">,</span> <span class="o">...</span><span class="p">,</span> <span class="m">255</span><span class="p">]},</span>
<span class="ss">restart:</span> <span class="ss">:permanent</span><span class="p">,</span>
<span class="ss">shutdown:</span> <span class="m">5000</span><span class="p">,</span>
<span class="ss">type:</span> <span class="ss">:worker</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Parameters are described <a href="https://hexdocs.pm/elixir/Supervisor.html#module-child-specification">here</a>,
so don’t think that it’s necessary to repeat that again.</p>
<p>You see here thee crazy brackets inside <code class="highlighter-rouge">start</code> tuple. So the question here is:</p>
<blockquote>
<p>Do I really need these brackets here? May be, if I have only one argument, I can ommit them?</p>
</blockquote>
<p>I don’t want to disappoint you, but these brackets have special sense, and should be here mandatory.</p>
<p>But I’ll try to explain you this brackets step by step.</p>
<h3 id="your-start_link-has-unknown-arity">Your start_link has unknown arity</h3>
<p>The first step is easy: <code class="highlighter-rouge">start_link</code> in your module has unknown arity
(unknown at the moment of inventing child specification).</p>
<p>So, you should have and instrument to start <code class="highlighter-rouge">start_link</code> dynamically - with arity 1,
or may be 2, or may be 255. And here comes well-known Erlang’s approach…</p>
<h3 id="kernelapply">Kernel.apply</h3>
<p>Let’s look at the <code class="highlighter-rouge">Kernel.apply/3</code> specification:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">apply</span><span class="p">(</span><span class="ss">module:</span> <span class="n">atom</span><span class="p">,</span> <span class="ss">function_name:</span> <span class="n">atom</span><span class="p">,</span> <span class="ss">args:</span> <span class="p">[</span><span class="n">any</span><span class="p">])</span> <span class="p">::</span> <span class="n">any</span>
</code></pre></div></div>
<p>and examples:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">iex</span><span class="o">></span> <span class="no">Enum</span><span class="o">.</span><span class="n">reverse</span><span class="p">([</span><span class="m">1</span><span class="p">,</span> <span class="m">2</span><span class="p">,</span> <span class="m">3</span><span class="p">])</span>
<span class="p">[</span><span class="m">3</span><span class="p">,</span> <span class="m">2</span><span class="p">,</span> <span class="m">1</span><span class="p">]</span>
<span class="n">iex</span><span class="o">></span> <span class="n">apply</span><span class="p">(</span><span class="no">Enum</span><span class="p">,</span> <span class="ss">:reverse</span><span class="p">,</span> <span class="p">[[</span><span class="m">1</span><span class="p">,</span> <span class="m">2</span><span class="p">,</span> <span class="m">3</span><span class="p">]])</span>
<span class="p">[</span><span class="m">3</span><span class="p">,</span> <span class="m">2</span><span class="p">,</span> <span class="m">1</span><span class="p">]</span>
</code></pre></div></div>
<p>As you see, <code class="highlighter-rouge">apply/3</code> doesn’t know how many arguments your function has.
Thats why he asks you to put these arguments into list.
Event if you have <strong>one</strong> argument - this arguments still should be in brackets.
This is separately pointed in <code class="highlighter-rouge">apply/3</code><strong>typespec</strong>.</p>
<p>The most crazy moment appears, when you have only <em>one</em> argument, and it is a <strong>list</strong> (like in the example) -
you will have double brackets, and you can do nothing with this…</p>
<p>Ok, let’s return to our child specification. Lets take tuple under <code class="highlighter-rouge">start</code> key, and <em>apply</em> it’s elements as the arguments to <code class="highlighter-rouge">Kernel.apply</code>:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">iex</span><span class="o">></span> <span class="no">Kernel</span><span class="o">.</span><span class="n">apply</span><span class="p">(</span><span class="no">Foo</span><span class="p">,</span> <span class="ss">:start_link</span><span class="p">,</span> <span class="p">[</span><span class="m">1</span><span class="p">,</span> <span class="m">2</span><span class="p">,</span> <span class="m">3</span><span class="p">,</span> <span class="o">...</span> <span class="m">255</span><span class="p">])</span>
<span class="c1"># and it is the same as...</span>
<span class="n">iex</span><span class="o">></span> <span class="no">Foo</span><span class="o">.</span><span class="n">start_link</span><span class="p">(</span><span class="m">1</span><span class="p">,</span> <span class="m">2</span><span class="p">,</span> <span class="m">3</span><span class="p">,</span> <span class="o">...</span><span class="p">,</span> <span class="m">255</span><span class="p">)</span>
</code></pre></div></div>
<p>Do you see this? Yep, <strong>child specification</strong> does not invent anything new -
it just <em>copy-pastes</em> tuple into <code class="highlighter-rouge">Kernel.apply</code>.
Thats why you need this brackets - and nothing to say more.</p>
<p>Let’s for a moment return to the example from the beginning:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">Stack</span><span class="o">.</span><span class="n">child_spec</span><span class="p">([</span><span class="ss">:hello</span><span class="p">])</span>
<span class="c1">#=> %{</span>
<span class="ss">id:</span> <span class="no">Stack</span><span class="p">,</span>
<span class="ss">start:</span> <span class="p">{</span><span class="no">Stack</span><span class="p">,</span> <span class="ss">:start_link</span><span class="p">,</span> <span class="p">[[</span><span class="ss">:hello</span><span class="p">]]},</span>
<span class="ss">restart:</span> <span class="ss">:permanent</span><span class="p">,</span>
<span class="ss">shutdown:</span> <span class="m">5000</span><span class="p">,</span>
<span class="ss">type:</span> <span class="ss">:worker</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Here, as you see, <code class="highlighter-rouge">Stack.start_link/1</code> is expecting to get <strong>one</strong> argument,
and this argument is <strong>list</strong>. That’s why we need to put <code class="highlighter-rouge">:hello</code> into double brackets.</p>
<p>To sum up, one mnemonic advise:</p>
<blockquote>
<p>If you see the <strong>function</strong>, that is passed by it’s <strong>name</strong> - remove <strong>one</strong> pair of <strong>brackets</strong>!</p>
</blockquote>
<p>These brackets are used to pass arguments as <strong>list</strong>, so the notation will omit the brackets
and put these arguments as are into function call.</p>
<h2 id="child_spec1---dynamic-point-in-your-worker">child_spec/1 - dynamic point in your worker</h2>
<p>Do you remember functions that our <code class="highlighter-rouge">use GenServer</code> brings into module?</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">iex</span><span class="o">></span> <span class="k">defmodule</span> <span class="no">Foo</span><span class="p">,</span> <span class="k">do</span><span class="p">:</span> <span class="kn">use</span> <span class="no">GenServer</span>
<span class="n">iex</span><span class="o">></span> <span class="no">Foo</span><span class="o">.</span><span class="n">__info__</span><span class="p">(</span><span class="ss">:functions</span><span class="p">)</span>
<span class="p">[</span><span class="ss">child_spec:</span> <span class="m">1</span><span class="p">,</span> <span class="ss">code_change:</span> <span class="m">3</span><span class="p">,</span> <span class="ss">handle_call:</span> <span class="m">3</span><span class="p">,</span> <span class="ss">handle_cast:</span> <span class="m">2</span><span class="p">,</span> <span class="ss">handle_info:</span> <span class="m">2</span><span class="p">,</span> <span class="ss">init:</span> <span class="m">1</span><span class="p">,</span> <span class="ss">terminate:</span> <span class="m">2</span><span class="p">]</span>
</code></pre></div></div>
<p>Yep, something new, which can’t be found neither in Erlang,
nor in Elixir 1.4 and previous - <code class="highlighter-rouge">child_spec/1</code> function.</p>
<p>Let’s try to look into it’s generation’s source code to understand, what is this function for:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">spec</span> <span class="o">=</span> <span class="p">[</span>
<span class="ss">id:</span> <span class="n">opts</span><span class="p">[</span><span class="ss">:id</span><span class="p">]</span> <span class="o">||</span> <span class="bp">__MODULE__</span><span class="p">,</span>
<span class="ss">start:</span> <span class="no">Macro</span><span class="o">.</span><span class="n">escape</span><span class="p">(</span><span class="n">opts</span><span class="p">[</span><span class="ss">:start</span><span class="p">])</span> <span class="o">||</span> <span class="kn">quote</span><span class="p">(</span><span class="k">do</span><span class="p">:</span> <span class="p">{</span><span class="bp">__MODULE__</span><span class="p">,</span> <span class="ss">:start_link</span><span class="p">,</span> <span class="p">[</span><span class="n">arg</span><span class="p">]}),</span>
<span class="ss">restart:</span> <span class="n">opts</span><span class="p">[</span><span class="ss">:restart</span><span class="p">]</span> <span class="o">||</span> <span class="ss">:permanent</span><span class="p">,</span>
<span class="ss">shutdown:</span> <span class="n">opts</span><span class="p">[</span><span class="ss">:shutdown</span><span class="p">]</span> <span class="o">||</span> <span class="m">5000</span><span class="p">,</span>
<span class="ss">type:</span> <span class="ss">:worker</span>
<span class="p">]</span>
<span class="nv">@doc</span> <span class="no">false</span>
<span class="k">def</span> <span class="n">child_spec</span><span class="p">(</span><span class="n">arg</span><span class="p">)</span> <span class="k">do</span>
<span class="p">%{</span><span class="n">unquote_splicing</span><span class="p">(</span><span class="n">spec</span><span class="p">)}</span>
<span class="k">end</span>
</code></pre></div></div>
<p>As you see - nothing hard. It’s expecting that you will pass some arguments in <code class="highlighter-rouge">use GenServer</code> statement, to somehow change attitude of <strong>child specification</strong>. For example, you can do something like this:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">use</span> <span class="no">GenServer</span><span class="p">,</span> <span class="ss">restart:</span> <span class="ss">:transient</span>
</code></pre></div></div>
<p>to redefine <strong>Restart strategy</strong>.</p>
<p>The only interesting place for us here - <code class="highlighter-rouge">{__MODULE__, :start_link, [arg]}</code> under <code class="highlighter-rouge">start</code> key in <em>child spec</em>.
As you see, <code class="highlighter-rouge">arg</code> which we pass into <code class="highlighter-rouge">child_spec/1</code> function is <em>wrapped into brackets</em>
and is prepared to be pushed into <code class="highlighter-rouge">start_link/1</code> function, that is defined in the same module.</p>
<p>As we know now, these <strong>brackets will be omited</strong>, so the call will be:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">MyGenServer</span><span class="o">.</span><span class="n">start_link</span><span class="p">(</span><span class="n">arg</span><span class="p">)</span>
</code></pre></div></div>
<p>Obviously, you can redefine this <code class="highlighter-rouge">child_spec/1</code> function. But remember, that is has arity 1.
Thus, if you want to define everything dynamically - you should wrap your data into data structure:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="n">child_spec</span><span class="p">({</span><span class="n">id</span><span class="p">,</span> <span class="p">{</span><span class="n">_module</span><span class="p">,</span> <span class="n">_fun</span><span class="p">,</span> <span class="n">args</span><span class="p">}</span> <span class="o">=</span> <span class="n">start</span><span class="p">,</span> <span class="n">restart</span><span class="p">,</span> <span class="n">shutdown</span><span class="p">})</span> <span class="k">do</span>
<span class="p">%{</span>
<span class="ss">id:</span> <span class="n">id</span><span class="p">,</span>
<span class="o">...</span>
<span class="p">}</span>
<span class="k">end</span>
<span class="c1"># or</span>
<span class="k">def</span> <span class="n">child_spec</span><span class="p">([</span><span class="n">id</span><span class="p">,</span> <span class="p">{</span><span class="n">_module</span><span class="p">,</span> <span class="n">_fun</span><span class="p">,</span> <span class="n">args</span><span class="p">}</span> <span class="o">=</span> <span class="n">start</span><span class="p">,</span> <span class="n">restart</span><span class="p">,</span> <span class="n">shutdown</span><span class="p">])</span> <span class="k">do</span>
<span class="p">%{</span>
<span class="ss">id:</span> <span class="n">id</span>
<span class="o">...</span>
<span class="p">}</span>
<span class="k">end</span>
<span class="c1"># or (this function has no sence, but as example... )</span>
<span class="k">def</span> <span class="n">child_spec</span><span class="p">(%{</span><span class="ss">id:</span> <span class="n">id</span><span class="p">,</span> <span class="ss">start:</span> <span class="p">{</span><span class="n">_module</span><span class="p">,</span> <span class="n">_fun</span><span class="p">,</span> <span class="n">args</span><span class="p">}</span> <span class="o">=</span> <span class="n">start</span><span class="p">,</span> <span class="ss">restart:</span> <span class="n">restart</span><span class="p">,</span> <span class="ss">shutdown:</span> <span class="n">shutdown</span><span class="p">}</span> <span class="o">=</span> <span class="n">spec</span><span class="p">)</span> <span class="k">do</span>
<span class="n">spec</span>
<span class="k">end</span>
<span class="c1"># Never Ever Do This At Home</span>
<span class="k">def</span> <span class="n">child_spec</span><span class="p">(</span><span class="n">id</span><span class="p">,</span> <span class="p">{</span><span class="n">_module</span><span class="p">,</span> <span class="n">_fun</span><span class="p">,</span> <span class="n">args</span><span class="p">}</span> <span class="o">=</span> <span class="n">start</span><span class="p">,</span> <span class="n">restart</span><span class="p">,</span> <span class="n">shutdown</span><span class="p">)</span> <span class="k">do</span>
<span class="p">%{</span>
<span class="ss">id:</span> <span class="n">id</span>
<span class="o">...</span>
<span class="p">}</span>
<span class="k">end</span>
<span class="c1"># Everybody will try to call YourModule.child_spec/1,</span>
<span class="c1"># but you defined YourModule.child_spec/4.</span>
<span class="c1"># Your function does not override child_spec/1.</span>
</code></pre></div></div>
<p>And now we are on the finish line…</p>
<h2 id="supervisor-wants-to-call-your-worker---but-doesnt-know-how">Supervisor wants to call your worker - but doesn’t know how</h2>
<p>Starting your <em>GenServers</em> via <code class="highlighter-rouge">start_link</code> from the code - not the best idea.
We have perfect <strong>OTP</strong> framework, which forces us to start all processes under <strong><em>supervision tree</em></strong>.
It’s defined by list of workers in casual way:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">Supervisor</span><span class="o">.</span><span class="n">start_link</span><span class="p">([</span>
<span class="n">_worker1_</span><span class="p">,</span>
<span class="n">_worker2_</span><span class="p">,</span>
<span class="o">...</span>
<span class="n">_workerN_</span>
<span class="p">],</span> <span class="n">opts</span><span class="p">)</span>
</code></pre></div></div>
<p>So, defining these <em>workers</em> has three different approaches, that all comes at the end to the <strong>child spec</strong>. Let’s dig into them one by one:</p>
<ul>
<li>
<p>A map representing the child specification itself - such as the child specification map outlined in the previous section:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">Supervisor</span><span class="o">.</span><span class="n">start_link</span><span class="p">([</span>
<span class="p">%{</span>
<span class="ss">id:</span> <span class="sd">"</span><span class="s2">id"</span><span class="p">,</span>
<span class="ss">start:</span> <span class="p">{</span><span class="no">MyModule</span><span class="p">,</span> <span class="ss">:start_link</span><span class="p">,</span> <span class="p">[</span><span class="no">true</span><span class="p">]},</span>
<span class="ss">restart:</span> <span class="ss">:transient</span><span class="p">,</span>
<span class="ss">shutdown:</span> <span class="m">500</span><span class="p">,</span>
<span class="ss">type:</span> <span class="n">worker</span>
<span class="p">}</span>
<span class="p">],</span> <span class="n">opts</span><span class="p">)</span>
</code></pre></div> </div>
<p>As you see, here <em>Supervisor</em> even doesn’t touch <code class="highlighter-rouge">YourModule.child_spec/1</code> function -
it’s starting supervised proces directly from the spec.
So, even you found perfect library in <em>Hex</em>, but it’s main <em>GenServer’s</em> <code class="highlighter-rouge">child_spec/1</code> is done awefull,
you still can adopt it into your <em>supervision tree</em> using this approach.</p>
</li>
<li>
<p>A tuple with a module as first element and the start argument as second:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">Supervisor</span><span class="o">.</span><span class="n">start_link</span><span class="p">([</span>
<span class="p">{</span><span class="no">MyModule</span><span class="p">,</span> <span class="no">true</span><span class="p">}</span>
<span class="p">],</span> <span class="n">opts</span><span class="p">)</span>
</code></pre></div> </div>
<p>When such format is used, the <strong>Supervisor</strong> will retrieve the <strong>child specification</strong> from <strong>MyModule</strong>.
Do you remember, that <code class="highlighter-rouge">child_spec/1</code> has arity 1? This is a place, when this arity helps:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="no">MyModule</span><span class="p">,</span> <span class="no">true</span><span class="p">}</span>
<span class="c1"># inside Supervisor initialization process this turn into</span>
<span class="no">MyModule</span><span class="o">.</span><span class="n">child_spec</span><span class="p">(</span><span class="no">true</span><span class="p">)</span>
<span class="c1"># Bad example</span>
<span class="p">{</span><span class="no">MyModule</span><span class="p">,</span> <span class="p">[</span><span class="no">true</span><span class="p">]}</span>
<span class="c1"># inside Supervisor initialization process this turn into</span>
<span class="no">MyModule</span><span class="o">.</span><span class="n">child_spec</span><span class="p">([</span><span class="no">true</span><span class="p">])</span>
<span class="c1"># and probably this is not what you want...</span>
</code></pre></div> </div>
</li>
<li>
<p>A module:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">Supervisor</span><span class="o">.</span><span class="n">start_link</span><span class="p">([</span>
<span class="no">MyModule</span>
<span class="p">],</span> <span class="n">opts</span><span class="p">)</span>
</code></pre></div> </div>
<p>In this case, it is equivalent to passing <code class="highlighter-rouge">{MyModule, []}</code>:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">MyModule</span>
<span class="c1"># inside Supervisor initialization process this turn into</span>
<span class="no">MyModule</span><span class="o">.</span><span class="n">child_spec</span><span class="p">([])</span>
</code></pre></div> </div>
</li>
</ul>
<h2 id="supervisorstart_child---one-more-tricky-point">Supervisor.start_child - one more tricky point.</h2>
<p>Generally saying, you have one more way to start worker under <strong>Supervisor</strong> - using <code class="highlighter-rouge">start_child/2</code>.</p>
<p>Let’s see it’s specification:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">start_child</span><span class="p">(</span>
<span class="ss">supervisor:</span> <span class="n">supervisor</span><span class="p">,</span>
<span class="ss">child_spec_or_args:</span> <span class="ss">:supervisor</span><span class="o">.</span><span class="n">child_spec</span> <span class="o">|</span> <span class="p">[</span><span class="n">term</span><span class="p">]</span>
<span class="p">)</span> <span class="p">::</span> <span class="n">on_start_child</span>
</code></pre></div></div>
<p>In specification we see interesting variable name: <code class="highlighter-rouge">child_spec_or_args</code>. What does that mean?
Maybe, we can start our child <strong>either</strong> with <em>child spec</em>, or with args?</p>
<p>The answer is - <strong>NO</strong>. And the second argument of <code class="highlighter-rouge">start_child/2</code> function will depends on the <strong>Supervisor’s</strong> strategy:</p>
<ul>
<li>if strategy is <code class="highlighter-rouge">:simple_one_for_one</code> - you should pass args</li>
<li>if strategy is <em>not</em> <code class="highlighter-rouge">:simple_one_for_one</code> - you should pass <em>child spec</em></li>
</ul>
<p>Why? Let’s dig into it!</p>
<h3 id="not-simple_one_for_one">NOT :simple_one_for_one</h3>
<p>We are defining <em>child spec</em> at the same time when we starting the worker.
That’s why we simply pass <em>child spec</em> - and new worker starts to work!</p>
<h3 id="simple_one_for_one">:simple_one_for_one</h3>
<p>With <code class="highlighter-rouge">:simple_one_for_one</code> strategy, we define <em>child spec</em> for our children at the same time,
when we initialize the <strong>Supervisor</strong>, but children are started <em>dynamically</em>. Here the problems come.</p>
<p>At the time when the <strong>Supervisor</strong> is initializing, we don’t know what <em>args</em> will be required by different workers,
that will be started in the future.</p>
<p>But this arguments in the form of <strong>list</strong> can be passed using <code class="highlighter-rouge">start_child/2</code> function!
This function will append these arguments to the <strong>existing</strong> arguments in predefined <em>child spec</em> -
just appending two lists <code class="highlighter-rouge">list1++list2</code>:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># For example our childspec is defined like this:</span>
<span class="p">%{</span>
<span class="ss">id:</span> <span class="sd">"</span><span class="s2">id"</span><span class="p">,</span>
<span class="ss">start:</span> <span class="p">{</span><span class="no">MyModule</span><span class="p">,</span> <span class="ss">:start_link</span><span class="p">,</span> <span class="p">[</span><span class="no">true</span><span class="p">]},</span>
<span class="ss">restart:</span> <span class="ss">:transient</span><span class="p">,</span>
<span class="ss">shutdown:</span> <span class="m">500</span><span class="p">,</span>
<span class="ss">type:</span> <span class="n">worker</span>
<span class="p">}</span>
<span class="c1"># We don't want to start our child, so we need to override our start</span>
<span class="c1"># part, and put there 0 arguments:</span>
<span class="n">spec</span> <span class="o">=</span> <span class="no">Supervisor</span><span class="o">.</span><span class="n">child_spec</span><span class="p">(</span><span class="no">MyModule</span><span class="p">,</span> <span class="ss">start:</span> <span class="p">{</span><span class="no">MyModule</span><span class="p">,</span> <span class="ss">:start_link</span><span class="p">,</span> <span class="p">[]})</span>
<span class="c1"># After this manipulation, child spec inside `spec` variable will be:</span>
<span class="p">%{</span>
<span class="ss">id:</span> <span class="sd">"</span><span class="s2">id"</span><span class="p">,</span>
<span class="ss">start:</span> <span class="p">{</span><span class="no">MyModule</span><span class="p">,</span> <span class="ss">:start_link</span><span class="p">,</span> <span class="p">[]},</span>
<span class="ss">restart:</span> <span class="ss">:transient</span><span class="p">,</span>
<span class="ss">shutdown:</span> <span class="m">500</span><span class="p">,</span>
<span class="ss">type:</span> <span class="n">worker</span>
<span class="p">}</span>
<span class="c1"># Then we are starting our supervisor:</span>
<span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="n">pid</span><span class="p">}</span> <span class="o">=</span> <span class="no">Supervisor</span><span class="o">.</span><span class="n">start_link</span><span class="p">([</span><span class="n">spec</span><span class="p">],</span> <span class="ss">strategy:</span> <span class="ss">:simple_one_for_one</span><span class="p">)</span>
<span class="c1"># The worker will not be started, because `start_link` with 0 args is not defined</span>
<span class="c1"># Now, starting worker dynamically:</span>
<span class="no">Supervisor</span><span class="o">.</span><span class="n">start_child</span><span class="p">(</span><span class="n">pid</span><span class="p">,</span> <span class="p">[</span><span class="no">true</span><span class="p">])</span> <span class="c1"># we have brackets here, because it's list!</span>
<span class="c1"># New child spec will be</span>
<span class="p">%{</span>
<span class="ss">id:</span> <span class="sd">"</span><span class="s2">id"</span><span class="p">,</span>
<span class="ss">start:</span> <span class="p">{</span><span class="no">MyModule</span><span class="p">,</span> <span class="ss">:start_link</span><span class="p">,</span> <span class="p">[]</span><span class="o">++</span><span class="p">[</span><span class="no">true</span><span class="p">]},</span> <span class="c1"># or simply [true]</span>
<span class="ss">restart:</span> <span class="ss">:transient</span><span class="p">,</span>
<span class="ss">shutdown:</span> <span class="m">500</span><span class="p">,</span>
<span class="ss">type:</span> <span class="n">worker</span>
<span class="p">}</span>
<span class="c1"># And new worker will start as we want</span>
</code></pre></div></div>
<h2 id="lets-get-some-examples">Let’s get some examples</h2>
<p>Just to summarize - let’s follow full <em>args waterfall</em> with interesting conditions:</p>
<ul>
<li>
<p>The easiest one: I have the most simple <strong>GenServer</strong> without any usefull state -
and just want to write as little amount of code as possible:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># In supervisor:</span>
<span class="no">Supervisor</span><span class="o">.</span><span class="n">start_link</span><span class="p">([</span><span class="no">SimpleModule</span><span class="p">],</span> <span class="n">opts</span><span class="p">)</span>
<span class="c1"># This will call child_spec this way:</span>
<span class="no">SimpleModule</span><span class="o">.</span><span class="n">child_spec</span><span class="p">([])</span>
<span class="c1"># I am to lazy to redefine child_spec, so the spec will be:</span>
<span class="p">%{</span>
<span class="ss">id:</span> <span class="no">SimpleModule</span><span class="p">,</span>
<span class="ss">start:</span> <span class="p">{</span><span class="no">SimpleModule</span><span class="p">,</span> <span class="ss">:start_link</span><span class="p">,</span> <span class="p">[[]]},</span> <span class="c1"># double brackets, but you already know why!</span>
<span class="ss">restart:</span> <span class="ss">:permanent</span><span class="p">,</span>
<span class="ss">shutdown:</span> <span class="m">5000</span><span class="p">,</span>
<span class="ss">type:</span> <span class="ss">:worker</span>
<span class="p">}</span>
<span class="c1"># Using this child spec, Supervisor will start our server this way:</span>
<span class="no">SimpleModule</span><span class="o">.</span><span class="n">start_link</span><span class="p">([])</span>
<span class="c1"># I am to lazy to think about args, so I'm deciding to bypass it into init through start_link,</span>
<span class="c1"># and defined start_link like this:</span>
<span class="k">def</span> <span class="n">start_link</span><span class="p">(</span><span class="n">arg</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="no">GenServer</span><span class="o">.</span><span class="n">start_link</span><span class="p">(</span><span class="bp">__MODULE__</span><span class="p">,</span> <span class="n">arg</span><span class="p">)</span>
<span class="c1"># Well, I'm not redefininig init also, so it will be called like this:</span>
<span class="no">SimpleModule</span><span class="o">.</span><span class="n">init</span><span class="p">([])</span>
<span class="c1"># and will start server with state:</span>
<span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="p">[]}</span>
</code></pre></div> </div>
</li>
<li>
<p>I don’t like name <code class="highlighter-rouge">start_link</code>, and want to start my <strong>GenServer</strong> using <code class="highlighter-rouge">star_blink</code>:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># In supervisor:</span>
<span class="no">Supervisor</span><span class="o">.</span><span class="n">start_link</span><span class="p">([</span><span class="no">AstronomyModule</span><span class="p">],</span> <span class="n">opts</span><span class="p">)</span>
<span class="c1"># This will call child_spec this way:</span>
<span class="no">AstronomyModule</span><span class="o">.</span><span class="n">child_spec</span><span class="p">([])</span>
<span class="c1"># Have to redefine child_spec here, to tell supervisor how to start my module:</span>
<span class="k">def</span> <span class="n">child_spec</span><span class="p">(</span><span class="n">arg</span><span class="p">)</span> <span class="k">do</span>
<span class="p">%{</span>
<span class="ss">id:</span> <span class="bp">__MODULE__</span><span class="p">,</span>
<span class="ss">start:</span> <span class="p">{</span><span class="bp">__MODULE__</span><span class="p">,</span> <span class="ss">:star_blink</span><span class="p">,</span> <span class="p">[</span><span class="n">arg</span><span class="p">]},</span>
<span class="ss">restart:</span> <span class="ss">:permanent</span><span class="p">,</span>
<span class="ss">shutdown:</span> <span class="m">5000</span><span class="p">,</span>
<span class="ss">type:</span> <span class="ss">:worker</span>
<span class="p">}</span>
<span class="k">end</span>
<span class="c1"># This spec will return:</span>
<span class="p">%{</span>
<span class="ss">id:</span> <span class="no">AstronomyModule</span><span class="p">,</span>
<span class="ss">start:</span> <span class="p">{</span><span class="no">AstronomyModule</span><span class="p">,</span> <span class="ss">:star_blink</span><span class="p">,</span> <span class="p">[[]]},</span> <span class="c1"># as in previous example</span>
<span class="ss">restart:</span> <span class="ss">:permanent</span><span class="p">,</span>
<span class="ss">shutdown:</span> <span class="m">5000</span><span class="p">,</span>
<span class="ss">type:</span> <span class="ss">:worker</span>
<span class="p">}</span>
<span class="c1"># Using this child spec, Supervisor will start our server this way:</span>
<span class="no">AstronomyModule</span><span class="o">.</span><span class="n">star_blink</span><span class="p">([])</span>
<span class="c1"># Bypassing arg into init through star_blink:</span>
<span class="k">def</span> <span class="n">star_blink</span><span class="p">(</span><span class="n">arg</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="no">GenServer</span><span class="o">.</span><span class="n">start_link</span><span class="p">(</span><span class="bp">__MODULE__</span><span class="p">,</span> <span class="n">arg</span><span class="p">)</span>
<span class="c1"># init will be called like this:</span>
<span class="no">AstronomyModule</span><span class="o">.</span><span class="n">init</span><span class="p">([])</span>
<span class="c1"># and will start server with state:</span>
<span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="p">[]}</span>
</code></pre></div> </div>
</li>
<li>
<p>I have <strong>list</strong> with <strong>one</strong> item as argument, but I didn’t read this article
and decided to add one more pair of brackets - just to be sure that everything will be all right:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># In supervisor:</span>
<span class="no">Supervisor</span><span class="o">.</span><span class="n">start_link</span><span class="p">([</span>
<span class="p">{</span><span class="no">BracketsModule</span><span class="p">,</span> <span class="p">[[</span><span class="ss">:element</span><span class="p">]]}</span>
<span class="p">],</span> <span class="n">opts</span><span class="p">)</span>
<span class="c1"># This will call child_spec this way:</span>
<span class="no">BracketsModule</span><span class="o">.</span><span class="n">child_spec</span><span class="p">([[</span><span class="ss">:element</span><span class="p">]])</span>
<span class="c1"># This spec will return:</span>
<span class="p">%{</span>
<span class="ss">id:</span> <span class="no">BracketsModule</span><span class="p">,</span>
<span class="ss">start:</span> <span class="p">{</span><span class="no">BracketsModule</span><span class="p">,</span> <span class="ss">:start_link</span><span class="p">,</span> <span class="p">[[[</span><span class="ss">:element</span><span class="p">]]]},</span> <span class="c1"># OH SHI~</span>
<span class="ss">restart:</span> <span class="ss">:permanent</span><span class="p">,</span>
<span class="ss">shutdown:</span> <span class="m">5000</span><span class="p">,</span>
<span class="ss">type:</span> <span class="ss">:worker</span>
<span class="p">}</span>
<span class="c1"># Using this child spec, Supervisor will start our server this way:</span>
<span class="no">BracketsModule</span><span class="o">.</span><span class="n">start_link</span><span class="p">([[</span><span class="ss">:element</span><span class="p">]])</span>
<span class="c1"># Bypassing arg into init through start_link:</span>
<span class="k">def</span> <span class="n">start_link</span><span class="p">(</span><span class="n">arg</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="no">GenServer</span><span class="o">.</span><span class="n">start_link</span><span class="p">(</span><span class="bp">__MODULE__</span><span class="p">,</span> <span class="n">arg</span><span class="p">)</span>
<span class="c1"># init will be called like this:</span>
<span class="no">AstronomyModule</span><span class="o">.</span><span class="n">init</span><span class="p">([[</span><span class="ss">:element</span><span class="p">]])</span>
<span class="c1"># and will start server with state:</span>
<span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="p">[[</span><span class="ss">:element</span><span class="p">]]}</span>
</code></pre></div> </div>
<p>Of course, we don’t want <em>list</em> in <em>list</em> as our state. So, we can try to solve the problem in the different ways:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># The only right way - change the root aka supervisor</span>
<span class="no">Supervisor</span><span class="o">.</span><span class="n">start_link</span><span class="p">([</span>
<span class="p">{</span><span class="no">BracketsModule</span><span class="p">,</span> <span class="p">[</span><span class="ss">:element</span><span class="p">]}</span>
<span class="p">],</span> <span class="n">opts</span><span class="p">)</span>
<span class="c1"># Never Ever Do This At Home</span>
<span class="c1"># Redefine child_spec:</span>
<span class="k">def</span> <span class="n">child_spec</span><span class="p">([</span><span class="n">arg</span><span class="p">])</span> <span class="k">do</span> <span class="c1"># trying to match brackets here</span>
<span class="p">%{</span>
<span class="ss">id:</span> <span class="bp">__MODULE__</span><span class="p">,</span>
<span class="ss">start:</span> <span class="p">{</span><span class="bp">__MODULE__</span><span class="p">,</span> <span class="ss">:star_blink</span><span class="p">,</span> <span class="p">[</span><span class="n">arg</span><span class="p">]},</span>
<span class="ss">restart:</span> <span class="ss">:permanent</span><span class="p">,</span>
<span class="ss">shutdown:</span> <span class="m">5000</span><span class="p">,</span>
<span class="ss">type:</span> <span class="ss">:worker</span>
<span class="p">}</span>
<span class="k">end</span>
<span class="c1"># Redefine start_link</span>
<span class="k">def</span> <span class="n">start_link</span><span class="p">([</span><span class="n">arg</span><span class="p">]),</span> <span class="k">do</span><span class="p">:</span> <span class="no">GenServer</span><span class="o">.</span><span class="n">start_link</span><span class="p">(</span><span class="bp">__MODULE__</span><span class="p">,</span> <span class="n">arg</span><span class="p">)</span>
<span class="c1"># Or redefine init</span>
<span class="k">def</span> <span class="n">init</span><span class="p">([</span><span class="n">arg</span><span class="p">]),</span> <span class="k">do</span><span class="p">:</span> <span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="n">arg</span><span class="p">}}</span>
</code></pre></div> </div>
</li>
</ul>
<h2 id="conclusion">Conclusion</h2>
<p>As you see, <strong>brackets hell</strong> is not so terrifying! From now you don’t need to restart your programs
because of silly mistakes with brackets in <strong>Supervisors</strong>, <strong>Specs</strong>, <strong>Start_links</strong> and <strong>Inits</strong>.</p>
<p>I’ll just try to give you last ideas explicetly - may be following these rules will help you and your team with communication and clearify your code:</p>
<ul>
<li>
<p>One should start architecturing <strong>from</strong> workers <strong>to</strong> supervisors.
<strong>From</strong> branches <strong>to</strong> roots. After all, <strong>workers</strong> will do the business job - they are main.
And, as you’ve seen in this article - you will have <strong>all</strong> instruments to start your workers properly -
doesn’t metter how <strong>workers</strong> are defined.
So, don’t make <em>data repacking</em> in <code class="highlighter-rouge">init</code> or <code class="highlighter-rouge">start_link</code> functions - pass necessary data directly from <strong>supervisor</strong></p>
</li>
<li>
<p>Try to follow <code class="highlighter-rouge">KISS</code> - don’t bring unnesessary complicity to your program.
For example, don’t redefine <code class="highlighter-rouge">init</code> in your <strong>GenServer</strong> if you can simply pass it’s initial state
from <code class="highlighter-rouge">start_link</code> function. And don’t redefine <code class="highlighter-rouge">child_spec</code> function,
if you are creating simple <strong>GenServer</strong> worker - better use arguments for <code class="highlighter-rouge">__using__</code> macro.</p>
</li>
<li>
<p>The main <em>thick</em> point of your <em>args flow</em> should be <code class="highlighter-rouge">child_spec</code> function. Even if everything is absolutely unique,
and not standard, try to make <code class="highlighter-rouge">init</code>, <code class="highlighter-rouge">start_link</code> as usual as possible from one side,
and try not to use <strong>child spec</strong> directly in <code class="highlighter-rouge">Supervisor.init</code> - from the other.
<code class="highlighter-rouge">child_spec</code> function can hold all the logic about <strong>how to start my worker</strong></p>
</li>
</ul>
<h2 id="acknowledgments">Acknowledgments</h2>
<p>Thanks to <a href="https://wunsh.ru/">@Wunsh</a> for translating this article to Russian! You can find the translation <a href="https://wunsh.ru/articles/elixir-brackets-hell-in-supervision-tree.html#subscribeModal">here</a>!</p>
<h2 id="the-end">The End</h2>The Supervisor.Spec module was deprecated and does not work with the module-based child specs, introduced in Elixir v1.5. Thus, all methods for supervision tree declaring were sugnificantly changed. It’s great time to understand the waterflow of passing arguments - from top-level Supervisor to low-level workers aka GenServers.Functional Rust: Cooking some beef!2017-05-23T00:00:00+00:002017-05-23T00:00:00+00:00https://virviil.github.io/2017/05/23/functional-rust-cooking-some-beef<p><img src="https://habrastorage.org/files/974/556/336/974556336a1f49a882e08295a72209c7.png" alt="Cow logo" class="img-responsive" /></p>
<p>One day I’ve stumbled upon Brainfuck-like language “Cow”. And suddenly I’ve came up with an idea to write an interpreter for it in new hip language Rust. Rust is an multi-paradigm language, which means you have to chose in which style you want to write your code in. I’ve chosen functional programming.
<!--more--></p>
<h2 id="concept">Concept</h2>
<p>State of our virtual [line-through]#cow# machine will be stored as immutable variable <code class="highlighter-rouge">state</code>. All actions will be made with functions like this:</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">fn</span> <span class="nf">change</span><span class="p">(</span><span class="n">previousState</span><span class="p">:</span> <span class="n">CowVM</span><span class="p">)</span> <span class="k">-></span> <span class="n">CowVM</span> <span class="c">// newState</span>
</code></pre></div></div>
<p>Looks a bit like <strong>Elm</strong>, <strong>Redux</strong>, and probably something else that I’m not aware of. Don’t you think so?</p>
<p>The fact that we know from the very beginning how we going to store our data makes this task a good target for functional programming.</p>
<p>Let’s look at the data our virtual machine stores:</p>
<ul>
<li>Program array - simple integer array;</li>
<li>Memory - second simple integer array;</li>
<li>Current command slot - index for value in program array;</li>
<li>Current memory position - index for value in memory array;</li>
<li>Register - almost simple integer value, why almost? Read further!</li>
<li>???</li>
<li><strong>PROFIT</strong></li>
</ul>
<p>From the very beginning I’m sure that at any point of development data will be store exactly like this. No additional <em>fields</em> or <em>views</em> will appear suddenly, no “just few little changes” from client. Living the dream!</p>
<h2 id="keeping-things-immutable">Keeping things immutable</h2>
<p>There is only one way to accomplish the task of keeping everything immutable - every time when we need to change anything we will create completely new state of program, saving it and disposing of old one. To accomplish this we will use magical features of <strong>Rust</strong>:</p>
<ul>
<li>Trait <strong>Copy</strong>. If you came from OOP, think of structures with this traits as of <em>value type</em> instead of <em>reference type</em>.</li>
<li>
<p>Magical <a href="https://doc.rust-lang.org/book/structs.html#update-syntax">struct update syntax</a>. It copies fields from old structure into new one.</p>
<p><strong>NOTE</strong>: It doesn’t actually copy fields that don’t have trait <strong>Copy</strong> (in this case <strong><code class="highlighter-rouge">Vec<i32></code></strong>).
It just moves them. And that’s very important,
because you can stumble upon errors like <a href="https://play.rust-lang.org/?code=%23%5Bderive(Clone%2C%20Debug)%5D%0Astruct%20Bar%20%7B%0A%20%20%20%20x%3A%20i32%2C%0A%20%20%20%20y%3A%20Vec%3Ci32%3E%0A%7D%0A%0Afn%20main()%20%7B%0A%20%20%20%20let%20a%20%3D%20Bar%7Bx%3A%201%2C%20y%3A%20vec!%5B1%2C2%2C3%5D%7D%3B%0A%20%20%20%20let%20b%20%3D%20Bar%7B..a%7D%3B%0A%20%20%20%20println!(%22%7B%3A%3F%7D%22%2C%20a.y)%3B%0A%7D&version=stable&backtrace=0">this</a>.
<strong>But</strong> not in our case! We don’t care about old structure and what happens to it.</p>
</li>
</ul>
<h2 id="model">Model</h2>
<p>Our code will have just one structure - the virtual cow machine itself:</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="err">#</span><span class="p">[</span><span class="nf">derive</span><span class="p">(</span><span class="n">Debug</span><span class="p">,</span> <span class="n">Default</span><span class="p">,</span> <span class="n">Clone</span><span class="p">)]</span>
<span class="k">pub</span> <span class="k">struct</span> <span class="n">CowVM</span> <span class="p">{</span>
<span class="k">pub</span> <span class="n">program</span><span class="p">:</span> <span class="nb">Vec</span><span class="o"><</span><span class="nb">u32</span><span class="o">></span><span class="p">,</span>
<span class="k">pub</span> <span class="n">memory</span><span class="p">:</span> <span class="nb">Vec</span><span class="o"><</span><span class="nb">i32</span><span class="o">></span><span class="p">,</span>
<span class="k">pub</span> <span class="n">program_position</span><span class="p">:</span> <span class="nb">usize</span><span class="p">,</span>
<span class="k">pub</span> <span class="n">memory_position</span><span class="p">:</span> <span class="nb">usize</span><span class="p">,</span>
<span class="k">pub</span> <span class="n">register</span><span class="p">:</span> <span class="nb">Option</span><span class="o"><</span><span class="nb">i32</span><span class="o">></span>
<span class="p">}</span>
</code></pre></div></div>
<p>Looks almost exactly like what I’ve described at concept phase, just adding few features -
<strong>Debug</strong>, <strong>Default</strong> and <strong>Clone</strong>. What it is and what it does you can read in my previous article.</p>
<h2 id="reducer">Reducer</h2>
<p>Nothing too complicated there. Just following documentation and writing functions for every command of the language.
These functions are taking and returning virtual machine, each time creating new state.</p>
<p>For example let’s look at very important command <strong>mOo</strong> - this command moves index of current memory slot back by 1:</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">pub</span> <span class="k">fn</span> <span class="nf">do_mOo</span><span class="p">(</span><span class="n">state</span><span class="p">:</span> <span class="n">CowVM</span><span class="p">)</span> <span class="k">-></span><span class="n">CowVM</span> <span class="p">{</span>
<span class="n">CowVM</span><span class="p">{</span>
<span class="n">memory_position</span><span class="p">:</span> <span class="n">state</span><span class="py">.memory_position</span><span class="err">-</span><span class="mi">1</span><span class="p">,</span>
<span class="n">program_position</span><span class="p">:</span> <span class="n">state</span><span class="py">.program_position</span><span class="o">+</span><span class="mi">1</span><span class="p">,</span>
<span class="o">..</span><span class="n">state</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>All that function does - it moves index of memory array back and moves index of program array forward.
Note that not all functions advance index of program array by 1, that’s why we don’t move this part to another function.
It doesn’t take more space in code than calling a function anyways.</p>
<p>Okay, let’s move to the interesting part. Remember me saying that register isn’t that simple?
That’s because the best (and probably the only) way to store <strong>nullable</strong> value in <strong>Rust</strong> is <a href="https://doc.rust-lang.org/std/option/"><strong>Option</strong></a> type.
It’s made in functional manner itself, by the way.
Let’s not get into <a href="https://en.wikipedia.org/wiki/Option_type">too much details</a>, just note that first of all,
this way is imposed by the language itself and second,
it’s radically different from every language out there with <strong>nil</strong>, <strong>null</strong> or anything similar.
These languages are usually called classic OOP languages: <strong>Java</strong>, <strong>C#</strong>, <strong>Python</strong>, <strong>Ruby</strong>, <strong>Go</strong>…
List goes on, but it’s not the point, just get used to the new order of things.</p>
<p>Anyways, let’s get back to our register. It can be empty and can be not empty, that’s why we use <strong>Option</strong>. And here is code for command that works with register:</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">pub</span> <span class="k">fn</span> <span class="nf">do_MMM</span><span class="p">(</span><span class="n">state</span><span class="p">:</span> <span class="n">CowVM</span><span class="p">)</span> <span class="k">-></span> <span class="n">CowVM</span> <span class="p">{</span>
<span class="k">match</span> <span class="n">state</span><span class="py">.register</span> <span class="p">{</span>
<span class="nf">Some</span><span class="p">(</span><span class="n">value</span><span class="p">)</span> <span class="k">=></span> <span class="p">{</span>
<span class="k">let</span> <span class="k">mut</span> <span class="n">memory</span> <span class="o">=</span> <span class="n">state</span><span class="py">.memory</span><span class="p">;</span>
<span class="n">memory</span><span class="p">[</span><span class="n">state</span><span class="py">.memory_position</span><span class="p">]</span> <span class="o">=</span> <span class="n">value</span><span class="p">;</span>
<span class="n">CowVM</span><span class="p">{</span>
<span class="n">register</span><span class="p">:</span> <span class="nb">None</span><span class="p">,</span>
<span class="n">program_position</span><span class="p">:</span> <span class="n">state</span><span class="py">.program_position</span><span class="o">+</span><span class="mi">1</span><span class="p">,</span>
<span class="n">memory</span><span class="p">:</span> <span class="n">memory</span><span class="p">,</span>
<span class="o">..</span><span class="n">state</span>
<span class="p">}</span>
<span class="p">},</span>
<span class="nb">None</span> <span class="k">=></span> <span class="p">{</span>
<span class="n">CowVM</span><span class="p">{</span>
<span class="n">register</span><span class="p">:</span> <span class="nf">Some</span><span class="p">(</span><span class="n">state</span><span class="py">.memory</span><span class="p">[</span><span class="n">state</span><span class="py">.memory_position</span><span class="p">]),</span>
<span class="n">program_position</span><span class="p">:</span> <span class="n">state</span><span class="py">.program_position</span><span class="o">+</span><span class="mi">1</span><span class="p">,</span>
<span class="o">..</span><span class="n">state</span>
<span class="p">}</span>
<span class="p">},</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Do you see these 4 closing curly braces at the end?
Terrifying. That’s why many functional languages aren’t using braces.</p>
<p>While we’re at it, let’s not ignore this not so elegant way to change value in memory,
maybe reader can suggest something better?
Let’s also note that in “pure functional” languages there are no arrays.
There are lists and dictionaries. Changing element in list takes <strong>O(N)</strong>, in dictionary - <strong>O(logN)</strong>, in our code we have <strong>O(1)</strong> and this is good. Regardless, data that looks like this:</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="s2">"0"</span><span class="p">:</span><span class="w"> </span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="s2">"1"</span><span class="p">:</span><span class="w"> </span><span class="mi">4</span><span class="p">,</span><span class="w"> </span><span class="err">...</span><span class="w"> </span><span class="p">,</span><span class="w"> </span><span class="s2">"255"</span><span class="p">:</span><span class="w"> </span><span class="mi">0</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>scares me greatly. So let it be.</p>
<p>Rest of the code we just write according to language specification. You can just look it up on github.</p>
<h2 id="main-cycle">Main cycle</h2>
<p>Very easy:</p>
<ul>
<li>Read file with source code;</li>
<li>Create new virtual machine with empty memory and filled program array;</li>
<li>Execute commands one by one until commands in program array end.</li>
</ul>
<p>Since we use <em>functional</em> way - everything has to be done recursively. So let’s do it.</p>
<p>Let’s define main recursive function - <strong>execute</strong>:</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">fn</span> <span class="nf">execute</span><span class="p">(</span><span class="n">state</span><span class="p">:</span> <span class="n">CowVM</span><span class="p">)</span> <span class="k">-></span> <span class="n">CowVM</span> <span class="p">{</span>
<span class="n">new_state</span> <span class="o">=</span> <span class="k">match</span> <span class="n">state</span><span class="py">.program</span><span class="p">[</span><span class="n">state</span><span class="py">.program_position</span><span class="p">]</span> <span class="p">{</span>
<span class="mi">0</span> <span class="k">=></span> <span class="nn">commands</span><span class="p">::</span><span class="nf">do_moo</span><span class="p">(</span><span class="n">state</span><span class="p">),</span>
<span class="mi">1</span> <span class="k">=></span> <span class="nn">commands</span><span class="p">::</span><span class="nf">do_mOo</span><span class="p">(</span><span class="n">state</span><span class="p">),</span>
<span class="mi">2</span> <span class="k">=></span> <span class="nn">commands</span><span class="p">::</span><span class="nf">do_moO</span><span class="p">(</span><span class="n">state</span><span class="p">),</span>
<span class="mi">3</span> <span class="k">=></span> <span class="nn">commands</span><span class="p">::</span><span class="nf">do_mOO</span><span class="p">(</span><span class="n">state</span><span class="p">),</span>
<span class="mi">4</span> <span class="k">=></span> <span class="nn">commands</span><span class="p">::</span><span class="nf">do_Moo</span><span class="p">(</span><span class="n">state</span><span class="p">),</span>
<span class="mi">5</span> <span class="k">=></span> <span class="nn">commands</span><span class="p">::</span><span class="nf">do_MOo</span><span class="p">(</span><span class="n">state</span><span class="p">),</span>
<span class="mi">6</span> <span class="k">=></span> <span class="nn">commands</span><span class="p">::</span><span class="nf">do_MoO</span><span class="p">(</span><span class="n">state</span><span class="p">),</span>
<span class="mi">7</span> <span class="k">=></span> <span class="nn">commands</span><span class="p">::</span><span class="nf">do_MOO</span><span class="p">(</span><span class="n">state</span><span class="p">),</span>
<span class="mi">8</span> <span class="k">=></span> <span class="nn">commands</span><span class="p">::</span><span class="nf">do_OOO</span><span class="p">(</span><span class="n">state</span><span class="p">),</span>
<span class="mi">9</span> <span class="k">=></span> <span class="nn">commands</span><span class="p">::</span><span class="nf">do_MMM</span><span class="p">(</span><span class="n">state</span><span class="p">),</span>
<span class="mi">10</span> <span class="k">=></span> <span class="nn">commands</span><span class="p">::</span><span class="nf">do_OOM</span><span class="p">(</span><span class="n">state</span><span class="p">),</span>
<span class="mi">11</span> <span class="k">=></span> <span class="nn">commands</span><span class="p">::</span><span class="nf">do_oom</span><span class="p">(</span><span class="n">state</span><span class="p">),</span>
<span class="n">_</span> <span class="k">=></span> <span class="n">state</span><span class="p">,</span>
<span class="p">}</span>
<span class="nf">execute</span><span class="p">(</span><span class="n">new_state</span><span class="p">);</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Simple logic - look up the new command, execute it and begin from the start. Keep doing it until nothing left to execute.</p>
<p>That’s it. Interpreter for language <strong>COW</strong> is done!</p>
<h2 id="real-main-cycle"><strong>Real</strong> main cycle</h2>
<p>Now you asking me - <strong>“What was that? Some kind of joke?”</strong>
The same question I asked myself when I’ve realized,
that “multi-paradigm” language <strong>Rust</strong> is lacking <strong>Tail Call Optimization</strong>.
(What is this, read <a href="http://stackoverflow.com/questions/310974/what-is-tail-call-optimization">here</a>.)</p>
<p>Without this feature you will quickly find out why site <strong>stackoverflow</strong> is <a href="https://is.gd/Tg5bid">called like that</a>.</p>
<p>Well, guess we will have to do cycle instead.</p>
<p>Let’s remove recursion from our <strong>execute</strong> function:</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">fn</span> <span class="nf">execute</span><span class="p">(</span><span class="n">state</span><span class="p">:</span> <span class="n">CowVM</span><span class="p">)</span> <span class="k">-></span> <span class="n">CowVM</span> <span class="p">{</span>
<span class="k">match</span> <span class="n">state</span><span class="py">.program</span><span class="p">[</span><span class="n">state</span><span class="py">.program_position</span><span class="p">]</span> <span class="p">{</span>
<span class="mi">0</span> <span class="k">=></span> <span class="nn">commands</span><span class="p">::</span><span class="nf">do_moo</span><span class="p">(</span><span class="n">state</span><span class="p">),</span>
<span class="mi">1</span> <span class="k">=></span> <span class="nn">commands</span><span class="p">::</span><span class="nf">do_mOo</span><span class="p">(</span><span class="n">state</span><span class="p">),</span>
<span class="mi">2</span> <span class="k">=></span> <span class="nn">commands</span><span class="p">::</span><span class="nf">do_moO</span><span class="p">(</span><span class="n">state</span><span class="p">),</span>
<span class="mi">3</span> <span class="k">=></span> <span class="nn">commands</span><span class="p">::</span><span class="nf">do_mOO</span><span class="p">(</span><span class="n">state</span><span class="p">),</span>
<span class="mi">4</span> <span class="k">=></span> <span class="nn">commands</span><span class="p">::</span><span class="nf">do_Moo</span><span class="p">(</span><span class="n">state</span><span class="p">),</span>
<span class="mi">5</span> <span class="k">=></span> <span class="nn">commands</span><span class="p">::</span><span class="nf">do_MOo</span><span class="p">(</span><span class="n">state</span><span class="p">),</span>
<span class="mi">6</span> <span class="k">=></span> <span class="nn">commands</span><span class="p">::</span><span class="nf">do_MoO</span><span class="p">(</span><span class="n">state</span><span class="p">),</span>
<span class="mi">7</span> <span class="k">=></span> <span class="nn">commands</span><span class="p">::</span><span class="nf">do_MOO</span><span class="p">(</span><span class="n">state</span><span class="p">),</span>
<span class="mi">8</span> <span class="k">=></span> <span class="nn">commands</span><span class="p">::</span><span class="nf">do_OOO</span><span class="p">(</span><span class="n">state</span><span class="p">),</span>
<span class="mi">9</span> <span class="k">=></span> <span class="nn">commands</span><span class="p">::</span><span class="nf">do_MMM</span><span class="p">(</span><span class="n">state</span><span class="p">),</span>
<span class="mi">10</span> <span class="k">=></span> <span class="nn">commands</span><span class="p">::</span><span class="nf">do_OOM</span><span class="p">(</span><span class="n">state</span><span class="p">),</span>
<span class="mi">11</span> <span class="k">=></span> <span class="nn">commands</span><span class="p">::</span><span class="nf">do_oom</span><span class="p">(</span><span class="n">state</span><span class="p">),</span>
<span class="n">_</span> <span class="k">=></span> <span class="n">state</span><span class="p">,</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>And start cycle in main function itself:</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">fn</span> <span class="nf">main</span><span class="p">()</span> <span class="p">{</span>
<span class="k">let</span> <span class="k">mut</span> <span class="n">state</span> <span class="o">=</span> <span class="nf">init_vm</span><span class="p">();</span>
<span class="k">loop</span> <span class="p">{</span>
<span class="k">if</span> <span class="n">state</span><span class="py">.program_position</span> <span class="o">==</span> <span class="n">state</span><span class="py">.program</span><span class="nf">.len</span><span class="p">()</span> <span class="p">{</span>
<span class="k">break</span><span class="p">;</span>
<span class="p">}</span>
<span class="n">state</span> <span class="o">=</span> <span class="nf">execute</span><span class="p">(</span><span class="n">state</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Do you feel all the pain of the world functional programming?
It’s not enough that this language made us forget our lovely recursions, it also made us use mutable variable!!!</p>
<p>We can’t do this:</p>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">fn</span> <span class="nf">main</span><span class="p">()</span> <span class="p">{</span>
<span class="k">let</span> <span class="n">state</span> <span class="o">=</span> <span class="nf">init_vm</span><span class="p">();</span>
<span class="k">loop</span> <span class="p">{</span>
<span class="k">if</span> <span class="n">state</span><span class="py">.program_position</span> <span class="o">==</span> <span class="n">state</span><span class="py">.program</span><span class="nf">.len</span><span class="p">()</span> <span class="p">{</span>
<span class="k">break</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">let</span> <span class="n">state</span> <span class="o">=</span> <span class="nf">execute</span><span class="p">(</span><span class="n">state</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>because of the reasons unknown? Actually because variable we’ve introduced in loop goes out of the scope and can’t be used afterwards.</p>
<h2 id="reading-moomoomoo-source-code">Reading MooMOOmOO source code</h2>
<p>There is nothing functional about working with <strong>IO</strong> in <strong>Rust</strong>. At all.
That’s why I’m not going to talk about it.
You can look at the source code yourself if you find it interesting.</p>
<h2 id="conclusion">Conclusion</h2>
<p>In my personal opinion, <strong>Rust</strong> language is already becoming <em>rusty</em> without even being old.
OOP doesn’t feel like OOP, FP doesn’t feel like FP.
What we left with is “multi-paradigm”.
Perhaps, exactly the mix of those paradigms will end up being cool!
Let’s hope. And write in <strong>Rust</strong>.</p>
<p>However, there are still reasons in following functional way. In whole program we managed to:</p>
<ul>
<li>Never go OOP and never create single class.</li>
<li>Never have problems with borrow checking. Never use references, muttable variables (almost),
never hearing word “ownership” from compiler. Let me tell you,
writing code knowing that it will always compile feels very good.</li>
<li>Never touch lifetimes parameters - here, the actual dark side of <strong>Rust</strong>.
Let me tell you - I’m afraid of all there <code class="highlighter-rouge">(x: &'a mut i32)</code> and I’m happy I didn’t have to do this.</li>
<li>Never make single trait. Not really great achievement, but apparently you don’t need traits in FP all that much.</li>
<li>Make all these functions “pure” and easy to test (maybe I will write about this,
but differences in testing OOP and FP are well known and easilly googlable).</li>
</ul>
<h2 id="afterword">Afterword</h2>
<p>Thank you all for finishing this. I welcome all of you in comment section, there we can discuss different paradigms in <strong>Rust</strong>. Any suggestions are also welcome.</p>
<p><a href="https://github.com/Virviil/cow.rs">Link to the source code</a></p>
<h2 id="acknowledgments">Acknowledgments</h2>
<p>Thanks to <a href="https://github.com/minizinger">@minizinger</a> for developing especially hard for me (being non-functional) parts of the code, inspiration and general help.</p>One day I’ve stumbled upon Brainfuck-like language “Cow”. And suddenly I’ve came up with an idea to write an interpreter for it in new hip language Rust. Rust is an multi-paradigm language, which means you have to chose in which style you want to write your code in. I’ve chosen functional programming.Elixir: Your package should survive in production!2016-11-30T00:00:00+00:002016-11-30T00:00:00+00:00https://virviil.github.io/2016/11/30/elixir-your-package-should-survive-in-production<p>You are very good person! Great person!
You have <strong>just created you first (or second, or third, etc.) package</strong> in Elixir language
and published it to the <a href="https://hex.pm"><em>Hex</em></a>.
You are nice enough to follow some conventions. At least <a href="https://hex.pm/docs/publish">this one</a>.</p>
<p>The work is done, isn’t it?
You’ve done everything as in <strong>official tutorial™</strong>. What problems can it lead to? “Nothing” is wrong answer…</p>
<p>Do you want to know why? Well… I’ll try to answer this question.
<!--more--></p>
<h2 id="your-mixexs-is-wrong">Your <code class="highlighter-rouge">mix.exs</code> is wrong!</h2>
<p>Of course it is. Let’s find some problems!</p>
<h3 id="build-consolidation-and-application-termination">Build consolidation and application termination</h3>
<p>Take a look at the function <code class="highlighter-rouge">project/0</code> in our <strong>Hex</strong> tutorial:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="n">project</span> <span class="k">do</span>
<span class="p">[</span><span class="ss">app:</span> <span class="ss">:postgrex</span><span class="p">,</span>
<span class="ss">version:</span> <span class="sd">"</span><span class="s2">0.1.0"</span><span class="p">,</span>
<span class="ss">elixir:</span> <span class="sd">"</span><span class="s2">0.13.0"</span><span class="p">,</span>
<span class="ss">description:</span> <span class="n">description</span><span class="p">(),</span>
<span class="ss">package:</span> <span class="n">package</span><span class="p">(),</span>
<span class="ss">deps:</span> <span class="n">deps</span><span class="p">()]</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Probably, you decided to follow this tutorial literally.
And deleted some incredibly important lines from <code class="highlighter-rouge">project/0</code> function.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">[</span><span class="ss">build_embedded:</span> <span class="no">Mix</span><span class="o">.</span><span class="n">env</span> <span class="o">==</span> <span class="ss">:prod</span><span class="p">,</span>
<span class="ss">start_permanent:</span> <span class="no">Mix</span><span class="o">.</span><span class="n">env</span> <span class="o">==</span> <span class="ss">:prod</span><span class="p">]</span>
</code></pre></div></div>
<p><code class="highlighter-rouge">:buid_embedded</code> tells our compiler to consolidate our project with all it’s dependencies
in <code class="highlighter-rouge">_build</code> directory.
Moreover it consolidates all protocols.
It’s not needed in you development environment, while all dependencies are installed on your working machine,
but imagine that you miss some of them on production server without <strong>Elixir</strong> or <strong>Erlang</strong> installation?</p>
<p><code class="highlighter-rouge">:start_permanent</code> tells your virtual machine to terminate after your application is terminated.
This is not necessary in development, because you need your <code class="highlighter-rouge">iex</code> to survive even when your application dies.
But in production you should supervise your application at the point of the whole virtual machine too.
So, it’s good when it crashes with your application.</p>
<p>What is application’s behaviour in production without these lines? Nobody knows.
May be OK. May be not. Just don’t forget to put them in your configuration, of course if they needed.</p>
<h3 id="whats-in-your-application0">What’s in your <code class="highlighter-rouge">application/0</code>?</h3>
<p>The second critical step. <code class="highlighter-rouge">application/0</code> function. In tutorial it’s empty:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="n">application</span> <span class="k">do</span>
<span class="p">[]</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Is it empty in your code?
Seems to me, that no. You have there <code class="highlighter-rouge">:logger</code>, maybe <code class="highlighter-rouge">:httpoison</code> or <code class="highlighter-rouge">:ecto</code>,
and without these deps your application even doesn’t start.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="n">application</span> <span class="k">do</span>
<span class="p">[</span><span class="ss">applications:</span> <span class="p">[</span><span class="ss">:logger</span><span class="p">,</span> <span class="ss">:httpoison</span><span class="p">,</span> <span class="ss">:gproc</span><span class="p">]]</span>
<span class="k">end</span>
</code></pre></div></div>
<p>The questions are:</p>
<ul>
<li>Should you add <code class="highlighter-rouge">mod</code> of your own application?</li>
<li>Or, maybe you should remove <code class="highlighter-rouge">:logger</code> from <code class="highlighter-rouge">:applications</code> list?</li>
</ul>
<p>This is the problem for your brains. You can solve it - but you should know, that this <strong>is</strong> a problem.</p>
<p>Don’t add <code class="highlighter-rouge">:mod</code> of your application in <code class="highlighter-rouge">application/0</code> function,
if the main module of your application doesn’t have <code class="highlighter-rouge">start_link</code> or <code class="highlighter-rouge">start</code> functions!
That is redundantly, and can cause bad effects in the future.</p>
<p>And of course, fix your <em>README</em>. Don’t force package users to add your application in <code class="highlighter-rouge">applications</code> list.
<strong>Mix</strong> will try to start you package before user’s application, but how will it perform that without <code class="highlighter-rouge">start</code> function?
Nobody knows.</p>
<p>The answer on the second question is simple - remove everything,
that is not needed to be <strong>started</strong> before your application.
<strong>Some packages</strong> are not <em>startable</em> - they doesn’t have <code class="highlighter-rouge">start</code> or <code class="highlighter-rouge">start_link</code> functions in application module.
How will <strong>Mix</strong> start them? You already know the answer.</p>
<hr />
<p><em><strong>Update</strong> (thanks to <a href="https://disqus.com/by/georgeguimaraes/">George Guimarães</a>)</em>:</p>
<p><strong>Elixir v1.4</strong> changed the way you should list deps.
Check it out <a href="https://github.com/elixir-lang/elixir/blob/v1.4/CHANGELOG.md#application-inference">here</a>.
You still have to think about applications that shouldn’t be started at runtime,
and list application from <em>OTP dist</em> that you use.</p>
<hr />
<h3 id="look-at-you-deps0">Look at you <code class="highlighter-rouge">deps/0</code>!</h3>
<p>I can bet that your package uses some dependencies.
They are defined in <code class="highlighter-rouge">deps/0</code> function, and of course they are up to date.
The problem is that then <strong>next day</strong> after you published the package <strong>your deps are NOT up to date!</strong>
Welcome to the reality!</p>
<p>You decided to use your friend Bob’s package, and this is your <code class="highlighter-rouge">deps/0</code> function:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defp</span> <span class="n">deps</span> <span class="k">do</span>
<span class="p">[{</span><span class="ss">:bobs_lib</span><span class="p">,</span> <span class="sd">"</span><span class="s2">= 0.2.1"</span><span class="p">,</span>
<span class="ss">:ex_doc</span><span class="p">,</span> <span class="ss">github:</span> <span class="sd">"</span><span class="s2">elixir-lang/ex_doc"</span><span class="p">}]</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Pretty good! But Bob unexpectedly released new bugfix with version <strong>v0.2.2</strong>.
And your friend Alex started to make package 2 days after you.
And this is his <code class="highlighter-rouge">deps/0</code> (he took data from <em>Hex</em> - no place for mistake!)</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defp</span> <span class="n">deps</span> <span class="k">do</span>
<span class="p">[{</span><span class="ss">:bobs_lib</span><span class="p">,</span> <span class="sd">"</span><span class="s2">= 0.2.2"</span><span class="p">,</span>
<span class="ss">:ex_doc</span><span class="p">,</span> <span class="ss">github:</span> <span class="sd">"</span><span class="s2">elixir-lang/ex_doc"</span><span class="p">}]</span>
<span class="k">end</span>
</code></pre></div></div>
<p>What would be with <strong>Mix</strong>, if it’s try to resolve <code class="highlighter-rouge">mix deps.get</code> in project with yours and Alex’s packages?</p>
<p>Yes… you already know the answer.</p>
<p>So. Please. Be careful with your deps versioning.
To be familiar with them - read <a href="http://semver.org/">here</a>…</p>
<p>But let’s return to Alex’s code:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defp</span> <span class="n">deps</span> <span class="k">do</span>
<span class="p">[{</span><span class="ss">:bobs_lib</span><span class="p">,</span> <span class="sd">"</span><span class="s2">= 0.2.2"</span><span class="p">,</span>
<span class="ss">:ex_doc</span><span class="p">,</span> <span class="ss">github:</span> <span class="sd">"</span><span class="s2">elixir-lang/ex_doc"</span><span class="p">}]</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Does he really needs this <strong>:ex_doc</strong> package in release? It’s just a junk in production.
It can increase deployment package size. But can’t increase anything else.</p>
<p>So, we can use it just in deployment with:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="ss">:ex_doc</span><span class="p">,</span> <span class="sd">"</span><span class="s2">~> 0.x.y"</span><span class="p">,</span> <span class="ss">only:</span> <span class="ss">:dev</span><span class="p">}</span>
</code></pre></div></div>
<p>Why don’t to do that, if we can? Just be patient to your package users - don’t force them
to pack all junk in there releases!</p>
<h2 id="going-deeper-hold-on-production-is-coming">Going deeper. Hold on: production is coming.</h2>
<p>Today the most common way to generate releases - <a href="https://github.com/bitwalker/distillery">Distillery</a>.
This is easy to use, up-to-date, and almost silver-bullet library,
that helps with making a candy from your <strong>Elixir</strong> application.</p>
<p>If you think that using <strong>Distillery</strong> has no sense, your users <strong>may think different</strong>.
They will try to build there applications, fail because of your package, and come to your <strong>Github’s Issues</strong> tab.
Ciao free time!</p>
<p>Of course, we can avoid this nerves.</p>
<h3 id="distillery-doesnt-know-your-erlang-deps">Distillery doesn’t know your Erlang deps!</h3>
<p>As you know, you can use Erlang modules in your code simply calling them via atom.
For example <code class="highlighter-rouge">:crypto.ec_curves</code>, <code class="highlighter-rouge">:observer.start</code> or <code class="highlighter-rouge">:calendar.local_time</code>.
While doing this operation, you don’t list <strong>Erlang</strong> modules as deps in your <code class="highlighter-rouge">deps/0</code> function.
And this is the anthill!</p>
<p><strong>Distillery</strong> doesn’t check the code of your application trying to find <strong>Erlang</strong> module calls.
It just consolidates your deps from <code class="highlighter-rouge">deps</code> and <code class="highlighter-rouge">:applications</code> list.
So your modules <code class="highlighter-rouge">:crypto</code> and <code class="highlighter-rouge">:calendar</code> <strong>would not be in release</strong>.
Will application work? Of course not. What is the solution to this problem?</p>
<p>Well, you should <strong>explicitly</strong> put all this <strong>Erlang</strong> deps in your <strong>:applications</strong> list.
Even if you knows that they are not <em>startable</em>. This hack is dirty. But it works.
And your package also works in production.</p>
<p><em>If you know better solution - use it and notify me in comments, so I can fix this article</em></p>
<h3 id="release-doesnt-have-mix-application">Release doesn’t have <code class="highlighter-rouge">Mix</code> application!</h3>
<p>Forget about custom <strong>Mix tasks</strong> and all functions from <strong>Mix</strong> module in your code.
They <strong>gonna fail in production</strong>.
There are some alternatives that you can read about in
<a href="https://hexdocs.pm/distillery/boot-hooks.html#content"><strong>Distillery</strong> docs</a>.</p>
<h3 id="your-configexs-compiles-into-release">Your <code class="highlighter-rouge">config.exs</code> compiles into release!</h3>
<p>Yes, it is.
You can’t change configuration after you released your program.
For example, you have different machines for <strong>:dev</strong>, <strong>:build</strong>, <strong>:test</strong> and <strong>:prod</strong> purposes.
They all have different configuration.
But after you build the release on your build machine… fairy-tale goes bad.
Config now is <strong>nailed</strong> to your release. So… what to do?</p>
<p>Well, obviously programmers are clever people.
And you know how to deal with that problem, aren’t you? <strong>Environment variables</strong>! And have all problems gone?… No</p>
<p>How do you read environments? Via <code class="highlighter-rouge">System.get_env</code>?
But values in <strong>config.exs</strong> are evaluated during the build time.
And <code class="highlighter-rouge">System.get_env</code> will be derived for your <strong>current</strong> environments!
The same with <code class="highlighter-rouge">Application.get_env</code>! What is the solution?</p>
<p>For first, don’t use <code class="highlighter-rouge">System.get_env</code> or <code class="highlighter-rouge">Application.get_env</code> in your config files.
There current values will be nailed.</p>
<p>Other clever people decided - we will use tuples for system variables like this:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">config</span> <span class="ss">:my_application</span><span class="p">,</span>
<span class="ss">my_key:</span> <span class="p">{</span><span class="ss">:system</span><span class="p">,</span> <span class="sd">"</span><span class="s2">ENV_VAR_NAME"</span><span class="p">}</span>
</code></pre></div></div>
<p>Thus, this tuple can be evaluated in runtime. Of course, you can configure some keys in old style way.</p>
<p>You can ask - but how is this connected to my package, if I don’t want to use and don’t use environment?
Well, look here:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">config</span> <span class="ss">:my_application</span><span class="p">,</span>
<span class="ss">my_key:</span> <span class="p">{</span><span class="ss">:system</span><span class="p">,</span> <span class="sd">"</span><span class="s2">ENV_VAR_NAME"</span><span class="p">}</span>
<span class="n">config</span> <span class="ss">:your_library</span><span class="p">,</span>
<span class="ss">your_key:</span> <span class="p">{</span><span class="ss">:system</span><span class="p">,</span> <span class="sd">"</span><span class="s2">ENV_YOUR_KEY_NAME"</span><span class="p">}</span>
</code></pre></div></div>
<p>What will be the result of <code class="highlighter-rouge">Application.get_env({:system, "ENV_YOUR_KEY_NAME"})</code> in release?
Yes… you know the answer.</p>
<p>So… what to do? <strong>Get prepared!</strong>. Because this approach has <em>backward compatibility</em> - implement it in your package!
And if once somebody very clever will try to use your package as a nunchaku - it will work!
Here is the <a href="https://github.com/Nebo15/confex">package</a> to help you
<em>(probably you can find some more with this functionality or create one by yourself)</em>.</p>
<p>And read about config in releases <a href="https://hexdocs.pm/distillery/runtime-configuration.html#content">here</a>.</p>
<h2 id="conclusion">Conclusion</h2>
<p>Well, my dear alchemist. I hope now you are ready to enter the <strong>Hex.pm</strong> hall of fame!
And don’t loose your free time removing curses from users of your package!</p>
<p>Read this article before each <code class="highlighter-rouge">mix hex.publish</code> execution.</p>
<p><em>Comment this article to help me keep it up-to date</em>.</p>
<p>Special thanks to <a href="https://github.com/noma4i">@noma4i</a> for pointing the area of the problem. Big ups.</p>You are very good person! Great person! You have just created you first (or second, or third, etc.) package in Elixir language and published it to the Hex. You are nice enough to follow some conventions. At least this one. The work is done, isn’t it? You’ve done everything as in official tutorial™. What problems can it lead to? “Nothing” is wrong answer… Do you want to know why? Well… I’ll try to answer this question.Elixir: Testing without starting supervision tree2016-10-26T00:00:00+00:002016-10-26T00:00:00+00:00https://virviil.github.io/2016/10/26/elixir-testing-without-starting-supervision-tree<p>Imagine simple situation: you need to test some of your code modules, but you application is made in the best qualities and traditions of OTP framework. When your tests start, your application starts also, and supervision tree starts also, and everything is running in the same environment with your tests!</p>
<p>Thus, you get a bench of problems:</p>
<ul>
<li>Your <strong>dev</strong> and <strong>test</strong> environments mix together;</li>
<li>You can’t test processes, that are registered with names, because they are already started by your supervision tree;</li>
<li>You can’t perform unit testing, because you are not really sure, that this testing would be really <em>unit</em>.</li>
</ul>
<p>The solution is simple: <strong>Divide and conquer!</strong>
<!--more-->
At the beginning - let’s turn of this annoying supervision tree!</p>
<h2 id="configure-your-mix-file">Configure your mix file</h2>
<p>The first thing is simple: dont start your application in test. Fortunately, <code class="highlighter-rouge">mix test</code> has special parameter: <code class="highlighter-rouge">--no-start</code>, which prevent any applications from starting in your test environment. Of course, we, developers, are so lazy - we don’t want to put this parameter every time when we are running tests. For more, imagine, that your team has a new member, who download the source, types <code class="highlighter-rouge">mix test</code>, and… get overkilled with tonnes of error logs. Not the funnies start on a new place, eh?</p>
<p>So, let’s modify aliases a bit. Change your <code class="highlighter-rouge">mix.exs</code> file in such a way:</p>
<hr />
<p><strong>.mix.exs</strong></p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="n">project</span> <span class="k">do</span>
<span class="p">[</span>
<span class="ss">app:</span> <span class="ss">:my_app</span><span class="p">,</span>
<span class="ss">version:</span> <span class="sd">"</span><span class="s2">0.1.0"</span><span class="p">,</span>
<span class="ss">elixir:</span> <span class="sd">"</span><span class="s2">~> 1.3"</span><span class="p">,</span>
<span class="ss">build_embedded:</span> <span class="no">Mix</span><span class="o">.</span><span class="n">env</span> <span class="o">==</span> <span class="ss">:prod</span><span class="p">,</span>
<span class="ss">start_permanent:</span> <span class="no">Mix</span><span class="o">.</span><span class="n">env</span> <span class="o">==</span> <span class="ss">:prod</span><span class="p">,</span>
<span class="ss">description:</span> <span class="n">description</span><span class="p">(),</span>
<span class="ss">package:</span> <span class="n">package</span><span class="p">(),</span>
<span class="ss">aliases:</span> <span class="n">aliases</span><span class="p">(),</span> <span class="c1">#(1)</span>
<span class="ss">deps:</span> <span class="n">deps</span><span class="p">()</span>
<span class="p">]</span>
<span class="k">end</span>
<span class="o">...</span>
<span class="k">defp</span> <span class="n">aliases</span> <span class="k">do</span>
<span class="p">[</span>
<span class="ss">test:</span> <span class="sd">"</span><span class="s2">test --no-start"</span> <span class="c1">#(2)</span>
<span class="p">]</span>
<span class="k">end</span>
</code></pre></div></div>
<ol>
<li>Here we adding new attribute - aliases - they are simple mix tasks. you can reed more <a href="http://elixir-lang.org/docs/stable/mix/Mix.html#module-aliases">here</a>.</li>
<li>Here we define your new alias. Change <code class="highlighter-rouge">test</code> to <code class="highlighter-rouge">test --no-start</code>, so when we run <code class="highlighter-rouge">mix test</code> it opens into <code class="highlighter-rouge">mix test --no-start</code>.</li>
</ol>
<hr />
<h2 id="configuring-your-exunit">Configuring your ExUnit</h2>
<p>Ok, now all applications are not started. But the question is:</p>
<blockquote>
<p>Is that what you need?</p>
</blockquote>
<p>Not only your application wasn’t started. You environment in working without <code class="highlighter-rouge">:logger</code>, <code class="highlighter-rouge">:kernel</code>, <code class="highlighter-rouge">:ecto</code>, and dozen applications, which are automatically started before your application. So… what to do?</p>
<p>The answer is: <strong>start them</strong>! The best place to do this - is before calling <code class="highlighter-rouge">ExUnit.start</code> in you <code class="highlighter-rouge">test_helper.exs</code> file. We can do this manually, if we want total control:</p>
<hr />
<p><strong>./test/test_helper.exs</strong></p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">Application</span><span class="o">.</span><span class="n">ensure_all_started</span><span class="p">(</span><span class="ss">:ecto</span><span class="p">)</span> <span class="c1">#(1)</span>
<span class="no">ExUnit</span><span class="o">.</span><span class="n">start</span><span class="p">()</span> <span class="c1">#(2)</span>
</code></pre></div></div>
<ol>
<li>Here we manually start <code class="highlighter-rouge">:ecto</code> application with all it’s dependencies. You can read more about this function <a href="http://elixir-lang.org/docs/stable/elixir/Application.html#ensure_all_started/2">here</a>.</li>
<li>This line is already in your autogenerated <code class="highlighter-rouge">test_helper.exs</code> file.</li>
</ol>
<hr />
<p>But this is the solution, how to start <strong>all</strong> dependencies of your application without starting your application:</p>
<hr />
<p><strong>./test/test_helper.exs</strong></p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">Application</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="ss">:my_app</span><span class="p">)</span> <span class="c1">#(1)</span>
<span class="n">for</span> <span class="n">app</span> <span class="o"><-</span> <span class="no">Application</span><span class="o">.</span><span class="n">spec</span><span class="p">(</span><span class="ss">:my_app</span><span class="p">,</span><span class="ss">:applications</span><span class="p">)</span> <span class="k">do</span> <span class="c1">#(2)</span>
<span class="no">Application</span><span class="o">.</span><span class="n">ensure_all_started</span><span class="p">(</span><span class="n">app</span><span class="p">)</span>
<span class="k">end</span>
<span class="no">ExUnit</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
</code></pre></div></div>
<ol>
<li>We should load application first, to get access to it’s specification.</li>
<li>List all the applications, which are dependencies of yours. Then just ensure, that they are all started. You can read more about <code class="highlighter-rouge">spec</code> function <a href="http://elixir-lang.org/docs/stable/elixir/Application.html#spec/2">here</a>.</li>
</ol>
<hr />
<p>Of course, if you will modify your dependencies, second solution will track this changes, in comparison with first, which forces you to control everything by yourself. In all cases, now you see, that you can write your user code to control what to start in your test cases.</p>
<h2 id="updates">Updates</h2>
<p>Thanks <a href="https://disqus.com/by/amberbit/">amberbit</a> for good comment about <code class="highlighter-rouge">Application.load</code> function!</p>Imagine simple situation: you need to test some of your code modules, but you application is made in the best qualities and traditions of OTP framework. When your tests start, your application starts also, and supervision tree starts also, and everything is running in the same environment with your tests! Thus, you get a bench of problems: Your dev and test environments mix together; You can’t test processes, that are registered with names, because they are already started by your supervision tree; You can’t perform unit testing, because you are not really sure, that this testing would be really unit. The solution is simple: Divide and conquer!