<![CDATA[JJ's blog]]>https://blog.jayjaydev.comhttps://cdn.hashnode.com/res/hashnode/image/upload/v1673609913110/TeU9FtOWn.pngJJ's bloghttps://blog.jayjaydev.comRSS for NodeTue, 15 Oct 2024 03:34:44 GMT60<![CDATA[Time Series and Forecasting with Python code examples, Part II]]>https://blog.jayjaydev.com/time-series-and-forecasting-with-python-code-examples-part-iihttps://blog.jayjaydev.com/time-series-and-forecasting-with-python-code-examples-part-iiFri, 23 Apr 2021 20:53:43 GMT<![CDATA[<p>We saw what a Time Series is and its main components in the previous post of this series on Time Series and Forecasting.</p><p>But we didn't talk anything about Forecasting!</p><p>Let's fix that. In this post, I'll introduce some very basic forecasting methods. They might seem too simple, but they are still used in practice. Furthermore, they are useful to build a baseline to compare more sophisticated methods with.</p><p>I strongly recommend this methodology of building simple baselines to evaluate further and more sophisticated models. Not just for Time Series Analysis but also in any branch of Machine Learning and Statistics.</p><p> So, let's get started!</p><h2 id="the-naive-method">The Naive Method</h2><p>Think about the dumbest method to forecast future values. </p><p>In just a minute you might discover the amazing Forecasting Naive Method! It just consists of predicting that the next value of the Series will be equal to the last recorded one.</p><p>For example, if our Series goes</p><p>$$X_1, X_1, ...., X_t$$</p><p>our next value will be </p><p>$$X_{t+1} = X_t$$</p><p>Pretty naive isn't it?</p><p>Actually, it can be good enough when the Series is a Random Walk (when the data don't follow a pattern and is messy). So, don't underestimate it.</p><p> I won't include any code in Python for this one because you can implement it in many ways and it is not necessary to use any third-party libraries.</p><h2 id="naive-seasonal-method">Naive Seasonal Method</h2><p>When the data shows a strong seasonal pattern, we can slightly improve the previous naive method.</p><p>Let's suppose we have monthly data, and we want to predict what the value for the next month (let's call it February) will be. Then we'll assume that value will be equal to the value shown for past February (the same month of the last year). Assuming that our data shows a strong monthly seasonal pattern.</p><p>More formally, if we define the seasonal period <code>p</code> as the number of samples between one observation and the next one that lies in the same season, we can predict</p><p>$$X_{t + h} = X_k $$</p><p>where \(k = t + h - p\).</p><p>Assuming that every value before \(X_{t + h}\) is defined.</p><p>Examples of values for the seasonal period <code>p</code> are:</p><ul><li>1, for <em>yearly</em> seasons</li><li>4, for <em>quarterly</em> seasons</li><li>12, for <em>monthly</em> seasons</li><li>52, for <em>weekly</em> seasons</li></ul><p>And so on.</p><p>This is a simple Python code example of the Naive Seasonal Method:</p><pre><code class="lang-python"><span class="hljs-keyword">import</span> numpy <span class="hljs-keyword">as</span> np<span class="hljs-keyword">import</span> pandas <span class="hljs-keyword">as</span> pd<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">seasonal_naive_forecast</span>(<span class="hljs-params">series, seasonal_period, forecast_horizon</span>):</span> <span class="hljs-keyword">if</span> len(series) < seasonal_period: <span class="hljs-keyword">raise</span> Error(<span class="hljs-string">"There must be at least 'seasonal_period' observations"</span>) <span class="hljs-comment"># We assume 'series' to be a pandas.Series object</span> prev_season = series.iloc[-seasonal_period:] season_number = int(np.ceil(forecast_horizon/seasonal_period)) next_seasons = np.tile(prev_season, season_number) <span class="hljs-comment"># We were only asked to predict the next 'forecast_horizon' values</span> <span class="hljs-keyword">return</span> pd.Series(next_seasons[:forecast_horizon])</code></pre><p>We improved the naive method to consider seasonality. Then, what about trending?</p><h2 id="drift-method">Drift method</h2><p>It would be great to allow the forecasting method to increase or decrease over time in cases when the data shows a trending behavior. That's what the drift method does.</p><p>The general formula would be:</p><p>$$X_{t+h} = X_t + h*\frac{Xt - X1}{t - 1}$$</p><p>Notice that the term $$\frac{Xt - X1}{t - 1}$$ is the slope of the line from the first observation to the last one. So, the drift method extrapolates that line into the future and assumes the data follows that trend.</p><p>The python code is omitted since it just consists of translating the previous formula to Python.</p><h2 id="conclusions">Conclusions</h2><p>In this post, we have explored the more basic forecasting methods. Although they are pretty simple, they might be good enough in many situations.</p><p>But remember that even when they were not a good solution, they are a great starting point to build our more sophisticated methods on top of.</p><p>In the upcoming posts of this series, we'll be talking about residual analysis, how to measure the performance of our models, and of course, how to build and use more robust and sophisticated models.</p><p>If you liked this post, consider subscribing to my newsletter so you don't miss an article from my blog. Do you prefer a simpler format and real-time interaction? Then, follow me on <a target="_blank" href="https://twitter.com/josejorgexl">Twitter</a>. I like to share more content there.</p><p>Stay tuned!</p><p>(Cover photo by <a href="https://unsplash.com/@javiestebaan?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Javier Esteban</a> on <a href="https://unsplash.com/s/photos/time-series?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a> )</p>]]><![CDATA[<p>We saw what a Time Series is and its main components in the previous post of this series on Time Series and Forecasting.</p><p>But we didn't talk anything about Forecasting!</p><p>Let's fix that. In this post, I'll introduce some very basic forecasting methods. They might seem too simple, but they are still used in practice. Furthermore, they are useful to build a baseline to compare more sophisticated methods with.</p><p>I strongly recommend this methodology of building simple baselines to evaluate further and more sophisticated models. Not just for Time Series Analysis but also in any branch of Machine Learning and Statistics.</p><p> So, let's get started!</p><h2 id="the-naive-method">The Naive Method</h2><p>Think about the dumbest method to forecast future values. </p><p>In just a minute you might discover the amazing Forecasting Naive Method! It just consists of predicting that the next value of the Series will be equal to the last recorded one.</p><p>For example, if our Series goes</p><p>$$X_1, X_1, ...., X_t$$</p><p>our next value will be </p><p>$$X_{t+1} = X_t$$</p><p>Pretty naive isn't it?</p><p>Actually, it can be good enough when the Series is a Random Walk (when the data don't follow a pattern and is messy). So, don't underestimate it.</p><p> I won't include any code in Python for this one because you can implement it in many ways and it is not necessary to use any third-party libraries.</p><h2 id="naive-seasonal-method">Naive Seasonal Method</h2><p>When the data shows a strong seasonal pattern, we can slightly improve the previous naive method.</p><p>Let's suppose we have monthly data, and we want to predict what the value for the next month (let's call it February) will be. Then we'll assume that value will be equal to the value shown for past February (the same month of the last year). Assuming that our data shows a strong monthly seasonal pattern.</p><p>More formally, if we define the seasonal period <code>p</code> as the number of samples between one observation and the next one that lies in the same season, we can predict</p><p>$$X_{t + h} = X_k $$</p><p>where \(k = t + h - p\).</p><p>Assuming that every value before \(X_{t + h}\) is defined.</p><p>Examples of values for the seasonal period <code>p</code> are:</p><ul><li>1, for <em>yearly</em> seasons</li><li>4, for <em>quarterly</em> seasons</li><li>12, for <em>monthly</em> seasons</li><li>52, for <em>weekly</em> seasons</li></ul><p>And so on.</p><p>This is a simple Python code example of the Naive Seasonal Method:</p><pre><code class="lang-python"><span class="hljs-keyword">import</span> numpy <span class="hljs-keyword">as</span> np<span class="hljs-keyword">import</span> pandas <span class="hljs-keyword">as</span> pd<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">seasonal_naive_forecast</span>(<span class="hljs-params">series, seasonal_period, forecast_horizon</span>):</span> <span class="hljs-keyword">if</span> len(series) < seasonal_period: <span class="hljs-keyword">raise</span> Error(<span class="hljs-string">"There must be at least 'seasonal_period' observations"</span>) <span class="hljs-comment"># We assume 'series' to be a pandas.Series object</span> prev_season = series.iloc[-seasonal_period:] season_number = int(np.ceil(forecast_horizon/seasonal_period)) next_seasons = np.tile(prev_season, season_number) <span class="hljs-comment"># We were only asked to predict the next 'forecast_horizon' values</span> <span class="hljs-keyword">return</span> pd.Series(next_seasons[:forecast_horizon])</code></pre><p>We improved the naive method to consider seasonality. Then, what about trending?</p><h2 id="drift-method">Drift method</h2><p>It would be great to allow the forecasting method to increase or decrease over time in cases when the data shows a trending behavior. That's what the drift method does.</p><p>The general formula would be:</p><p>$$X_{t+h} = X_t + h*\frac{Xt - X1}{t - 1}$$</p><p>Notice that the term $$\frac{Xt - X1}{t - 1}$$ is the slope of the line from the first observation to the last one. So, the drift method extrapolates that line into the future and assumes the data follows that trend.</p><p>The python code is omitted since it just consists of translating the previous formula to Python.</p><h2 id="conclusions">Conclusions</h2><p>In this post, we have explored the more basic forecasting methods. Although they are pretty simple, they might be good enough in many situations.</p><p>But remember that even when they were not a good solution, they are a great starting point to build our more sophisticated methods on top of.</p><p>In the upcoming posts of this series, we'll be talking about residual analysis, how to measure the performance of our models, and of course, how to build and use more robust and sophisticated models.</p><p>If you liked this post, consider subscribing to my newsletter so you don't miss an article from my blog. Do you prefer a simpler format and real-time interaction? Then, follow me on <a target="_blank" href="https://twitter.com/josejorgexl">Twitter</a>. I like to share more content there.</p><p>Stay tuned!</p><p>(Cover photo by <a href="https://unsplash.com/@javiestebaan?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Javier Esteban</a> on <a href="https://unsplash.com/s/photos/time-series?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a> )</p>]]>https://cdn.hashnode.com/res/hashnode/image/upload/v1619209070952/rUSZ-do17.jpeg<![CDATA[Time Series and Forecasting with Python code examples (Part I)]]>https://blog.jayjaydev.com/time-series-and-forecasting-with-python-code-examples-part-ihttps://blog.jayjaydev.com/time-series-and-forecasting-with-python-code-examples-part-iTue, 13 Apr 2021 23:46:40 GMT<![CDATA[<p>A Time Series is just a set of data collected in a certain time span. For example, the weekly values of some stocks, and the annual average precipitation indices in a country. They are widely used to predict future events like the expenses of next month, or the number of hurricanes in the Atlantic ocean for the upcoming season.</p><p>What do those examples have that makes them different from other datasets?</p><p>There is an implicit order relationship in the data, we can't just shuffle the data and work with it. It is extremely important which registry was collected before and which one after.</p><p>Another important characteristic (that supports the previous one) is that we assume there is some relationship between previous entries and future ones. In other words, we can write a data sample as <code>Xn = f(Xn-1, Xn-2, ..., X1, S) + e</code>, where <code>S</code> represents other parameters that can be useful to predict the value of <code>Xn</code> and <code>e</code> represents a random error. We call such a relation in which a term can be determined as a function of the previous ones a <strong>recurrence</strong>.</p><p>This post is the first in a series about Time Series and Forecasting. Here, we are going to explain the main components of a Time Series, how to measure the relationships between previous and future data of the Time Series, and the first steps to analyze a forecasting problem.</p><p>I include a practical exercise to illustrate what we'll be talking about. All using Python.</p><p>What do you think? Let's get started!</p><h2 id="time-series-and-its-main-components">Time series and its main components</h2><p>Let's define the most important features to look at in a Time Series.</p><p>From the fact of being a sequence of observations during a time span, Time Series can present some patterns that determine the way we analyze them.</p><p>Those patterns are:</p><ul><li><p><strong>Seasonality</strong>: When the data is affected by some seasonal factors such as holidays, seasons of the year, sports events, and so on. Those seasonal effects have a fixed frequency.</p></li><li><p><strong>Trending</strong>: When the data seems to have a long-term increase or decrease in its values. Although most of the examples present this trending quality as a linear modification of the data, this increasing/decreasing behavior might not be linear.</p></li><li><p><strong>Cycling</strong>: When there are other external factors with no fixed frequency that affect the data during a long term. This could be due to economic conditions for example. This pattern is not equivalent to the Seasonality pattern. It usually has a longer period and, as I already said, its frequency is not fixed.</p></li></ul><p>The following image illustrates how all the previous patterns can appear in a Time Series. The image is taken from <a target="_blank" href="https://otexts.com/fpp2/">Forecasting: Principles and Practice</a>, which is an excellent resource to get started with Time Series. </p><p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1618344155384/FolELGcpy.png" alt="fourexamples-1.png" /></p><p>The top-left example shows a strong seasonality within each year, as well as some strong cyclic behavior with a period of about 610 years. There is no apparent trend in the data <strong>over this period</strong>.</p><p>The top-right example shows no seasonality, but an obvious downward trend.</p><p>The bottom-left example shows a strong increasing trend, with strong seasonality. There is no evidence of any cyclic behavior.</p><p>The bottom-right one has no trend, seasonality, or cyclic behavior. There are random fluctuations that do not appear to be very predictable, and no strong patterns that would help with developing a forecasting model.</p><p>It is important to note that the patterns identified above correspond with the time period that is represented in the graphs. If we had a longer period, other patterns could emerge.</p><h2 id="an-example">An example</h2><p>We'll be working with a dataset that represents the monthly sales of shampoo during three years. You can download it <a target="_blank" href="https://raw.githubusercontent.com/jbrownlee/Datasets/master/shampoo.csv">here</a></p><p>Let's explore the data!</p><pre><code class="lang-python"><span class="hljs-keyword">import</span> pandas <span class="hljs-keyword">as</span> pd<span class="hljs-keyword">import</span> matplotlib.pyplot <span class="hljs-keyword">as</span> pltshampoo_df = pd.read_csv(<span class="hljs-string">'shampoo.csv'</span>)print(shampoo_df.head())print(shampoo_df.shape)<span class="hljs-comment"># Month Sales</span><span class="hljs-comment"># 0 1-01 266.0</span><span class="hljs-comment"># 1 1-02 145.9</span><span class="hljs-comment"># 2 1-03 183.1</span><span class="hljs-comment"># 3 1-04 119.3</span><span class="hljs-comment"># 4 1-05 180.3</span><span class="hljs-comment">#</span><span class="hljs-comment"># (36, 2)</span></code></pre><p>As you can see, the dataset contains 36 rows corresponding to the sales of 36 consecutive months.</p><p>Let's see what patterns are present in our data:</p><pre><code class="lang-python">plt.plot(shampoo_df.Month, shampoo_df.Sales)plt.show()</code></pre><p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1618345453686/1P-CeShd3.png" alt="Figure_1.png" /></p><p>Here you can see a strong increasing trending and a perceptible seasonal pattern. There is no evidence of any cyclic behavior.</p><p>Cool! But... What's next?</p><h2 id="autocorrelation">Autocorrelation</h2><p>The way we use the components of a Time Series is left for the next delivery of this series. I want to talk about other important feature that allows us to determine how predictable might be the Series we are working on.</p><p>As I mentioned in the first part of this article, we hope there are some recurrent dependencies between the elements of the Series (i.e. we can predict future values from previous ones). So, it is important to measure the relationship between the elements of the Series.</p><p>In other statistical problems, we are usually interested in the <strong>correlation</strong> between some pair of variables. That gives us a measure of the strength of the linear relationship between those variables. Besides the correlation between two different variables, there is another metric that we are interested in when dealing with Time Series: <strong>autocorrelation</strong>.</p><p>It is a very similar concept but this one is used to measure the relationship that exists between the previous observations and the most recent ones. So, instead of measuring a linear relationship between two different variables, we measure the relationship between the values of the same variable through time.</p><p>We can define multiple autocorrelation coefficients. What determines a coefficient is the number of past observations that we use. For example, the coefficient <code>r2</code>, is a measure of the linear relationship between every sample and the previous one; <code>r3</code> takes into account the two previous observations, and so on. In general, the coefficient <code>rk</code> can be calculated as:</p><p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1618347318479/lK2Ry1BLi.png" alt="render.png" /></p><p>Then, we can plot every autocorrelation coefficient in what is called an <em>autocorrelogram</em>. Let's do it with our data!</p><pre><code class="lang-python">acorr = pd.plotting.autocorrelation_plot(shampoo_df.Sales)acorr.plot()plt.show()</code></pre><p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1618347765065/QTIeajn8L.png" alt="Figure_2.png" /></p><p>The continuous horizontal lines correspond to the values <code>2/sqrt(T)</code> and <code>-2/sqrt(T)</code>, and enclose a region with an insignificant strength in the linear relationship.</p><p>As you can see, there seems to be a strong linear relationship between the sales of a month and the sales of the previous one or two months, while there is not too much evidence of a linear relationship when we look at more than five lagged months.</p><h2 id="conclusions">Conclusions</h2><p>Let's stop here. So far we have seen what a Time Series is, what are the main components of a Time Series, how to interpret them, and how to determine if there is a linear relationship between lagged observations.</p><p>In the next post, we'll learn how to decompose a Time Series in its multiple components, how to determine the best models to explain a Time Series, and much more.</p><p>So, be sure to stay tuned! You can subscribe to my newsletter and receive a notification so you don't miss any of my posts.</p><p>Want to get my content in a simpler format and be able to interact in a different fashion? Follow me on Twitter!</p><p>See you soon!</p><p>(Cover photo by <a href="https://unsplash.com/@javiestebaan?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Javier Esteban</a> on <a href="https://unsplash.com/s/photos/time-series?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a> )</p>]]><![CDATA[<p>A Time Series is just a set of data collected in a certain time span. For example, the weekly values of some stocks, and the annual average precipitation indices in a country. They are widely used to predict future events like the expenses of next month, or the number of hurricanes in the Atlantic ocean for the upcoming season.</p><p>What do those examples have that makes them different from other datasets?</p><p>There is an implicit order relationship in the data, we can't just shuffle the data and work with it. It is extremely important which registry was collected before and which one after.</p><p>Another important characteristic (that supports the previous one) is that we assume there is some relationship between previous entries and future ones. In other words, we can write a data sample as <code>Xn = f(Xn-1, Xn-2, ..., X1, S) + e</code>, where <code>S</code> represents other parameters that can be useful to predict the value of <code>Xn</code> and <code>e</code> represents a random error. We call such a relation in which a term can be determined as a function of the previous ones a <strong>recurrence</strong>.</p><p>This post is the first in a series about Time Series and Forecasting. Here, we are going to explain the main components of a Time Series, how to measure the relationships between previous and future data of the Time Series, and the first steps to analyze a forecasting problem.</p><p>I include a practical exercise to illustrate what we'll be talking about. All using Python.</p><p>What do you think? Let's get started!</p><h2 id="time-series-and-its-main-components">Time series and its main components</h2><p>Let's define the most important features to look at in a Time Series.</p><p>From the fact of being a sequence of observations during a time span, Time Series can present some patterns that determine the way we analyze them.</p><p>Those patterns are:</p><ul><li><p><strong>Seasonality</strong>: When the data is affected by some seasonal factors such as holidays, seasons of the year, sports events, and so on. Those seasonal effects have a fixed frequency.</p></li><li><p><strong>Trending</strong>: When the data seems to have a long-term increase or decrease in its values. Although most of the examples present this trending quality as a linear modification of the data, this increasing/decreasing behavior might not be linear.</p></li><li><p><strong>Cycling</strong>: When there are other external factors with no fixed frequency that affect the data during a long term. This could be due to economic conditions for example. This pattern is not equivalent to the Seasonality pattern. It usually has a longer period and, as I already said, its frequency is not fixed.</p></li></ul><p>The following image illustrates how all the previous patterns can appear in a Time Series. The image is taken from <a target="_blank" href="https://otexts.com/fpp2/">Forecasting: Principles and Practice</a>, which is an excellent resource to get started with Time Series. </p><p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1618344155384/FolELGcpy.png" alt="fourexamples-1.png" /></p><p>The top-left example shows a strong seasonality within each year, as well as some strong cyclic behavior with a period of about 610 years. There is no apparent trend in the data <strong>over this period</strong>.</p><p>The top-right example shows no seasonality, but an obvious downward trend.</p><p>The bottom-left example shows a strong increasing trend, with strong seasonality. There is no evidence of any cyclic behavior.</p><p>The bottom-right one has no trend, seasonality, or cyclic behavior. There are random fluctuations that do not appear to be very predictable, and no strong patterns that would help with developing a forecasting model.</p><p>It is important to note that the patterns identified above correspond with the time period that is represented in the graphs. If we had a longer period, other patterns could emerge.</p><h2 id="an-example">An example</h2><p>We'll be working with a dataset that represents the monthly sales of shampoo during three years. You can download it <a target="_blank" href="https://raw.githubusercontent.com/jbrownlee/Datasets/master/shampoo.csv">here</a></p><p>Let's explore the data!</p><pre><code class="lang-python"><span class="hljs-keyword">import</span> pandas <span class="hljs-keyword">as</span> pd<span class="hljs-keyword">import</span> matplotlib.pyplot <span class="hljs-keyword">as</span> pltshampoo_df = pd.read_csv(<span class="hljs-string">'shampoo.csv'</span>)print(shampoo_df.head())print(shampoo_df.shape)<span class="hljs-comment"># Month Sales</span><span class="hljs-comment"># 0 1-01 266.0</span><span class="hljs-comment"># 1 1-02 145.9</span><span class="hljs-comment"># 2 1-03 183.1</span><span class="hljs-comment"># 3 1-04 119.3</span><span class="hljs-comment"># 4 1-05 180.3</span><span class="hljs-comment">#</span><span class="hljs-comment"># (36, 2)</span></code></pre><p>As you can see, the dataset contains 36 rows corresponding to the sales of 36 consecutive months.</p><p>Let's see what patterns are present in our data:</p><pre><code class="lang-python">plt.plot(shampoo_df.Month, shampoo_df.Sales)plt.show()</code></pre><p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1618345453686/1P-CeShd3.png" alt="Figure_1.png" /></p><p>Here you can see a strong increasing trending and a perceptible seasonal pattern. There is no evidence of any cyclic behavior.</p><p>Cool! But... What's next?</p><h2 id="autocorrelation">Autocorrelation</h2><p>The way we use the components of a Time Series is left for the next delivery of this series. I want to talk about other important feature that allows us to determine how predictable might be the Series we are working on.</p><p>As I mentioned in the first part of this article, we hope there are some recurrent dependencies between the elements of the Series (i.e. we can predict future values from previous ones). So, it is important to measure the relationship between the elements of the Series.</p><p>In other statistical problems, we are usually interested in the <strong>correlation</strong> between some pair of variables. That gives us a measure of the strength of the linear relationship between those variables. Besides the correlation between two different variables, there is another metric that we are interested in when dealing with Time Series: <strong>autocorrelation</strong>.</p><p>It is a very similar concept but this one is used to measure the relationship that exists between the previous observations and the most recent ones. So, instead of measuring a linear relationship between two different variables, we measure the relationship between the values of the same variable through time.</p><p>We can define multiple autocorrelation coefficients. What determines a coefficient is the number of past observations that we use. For example, the coefficient <code>r2</code>, is a measure of the linear relationship between every sample and the previous one; <code>r3</code> takes into account the two previous observations, and so on. In general, the coefficient <code>rk</code> can be calculated as:</p><p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1618347318479/lK2Ry1BLi.png" alt="render.png" /></p><p>Then, we can plot every autocorrelation coefficient in what is called an <em>autocorrelogram</em>. Let's do it with our data!</p><pre><code class="lang-python">acorr = pd.plotting.autocorrelation_plot(shampoo_df.Sales)acorr.plot()plt.show()</code></pre><p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1618347765065/QTIeajn8L.png" alt="Figure_2.png" /></p><p>The continuous horizontal lines correspond to the values <code>2/sqrt(T)</code> and <code>-2/sqrt(T)</code>, and enclose a region with an insignificant strength in the linear relationship.</p><p>As you can see, there seems to be a strong linear relationship between the sales of a month and the sales of the previous one or two months, while there is not too much evidence of a linear relationship when we look at more than five lagged months.</p><h2 id="conclusions">Conclusions</h2><p>Let's stop here. So far we have seen what a Time Series is, what are the main components of a Time Series, how to interpret them, and how to determine if there is a linear relationship between lagged observations.</p><p>In the next post, we'll learn how to decompose a Time Series in its multiple components, how to determine the best models to explain a Time Series, and much more.</p><p>So, be sure to stay tuned! You can subscribe to my newsletter and receive a notification so you don't miss any of my posts.</p><p>Want to get my content in a simpler format and be able to interact in a different fashion? Follow me on Twitter!</p><p>See you soon!</p><p>(Cover photo by <a href="https://unsplash.com/@javiestebaan?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Javier Esteban</a> on <a href="https://unsplash.com/s/photos/time-series?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a> )</p>]]>https://cdn.hashnode.com/res/hashnode/image/upload/v1618348455545/aXFeYEszZ.jpeg<![CDATA[Gödel's Theorems from the History to the Demonstrations]]>https://blog.jayjaydev.com/godels-theorems-from-the-history-to-the-demonstrationshttps://blog.jayjaydev.com/godels-theorems-from-the-history-to-the-demonstrationsThu, 18 Feb 2021 02:52:58 GMT<![CDATA[<p><em>This is a collection of three threads published on Twitter, about Goedel's Theorems.</em></p><h2 id="introduction">Introduction</h2><p>Today I want to start a potentially little series of threads about one of my biggest passions: Gdel's theorems. One of the greatest discoveries of all time that I'm sure will blow your mind! I'll structure the series as follows.</p><p>First: The history that preceded the theorems</p><p>Then: The theorems. Explanation and some common misconceptions</p><p>And finally: Shallow overview of the demonstrations. </p><p>Let's start with the history that preceded Gdel's results. It is pretty much the history of math so let's see if we can comprise it in a thread. </p><h2 id="the-history">The history</h2><p>The study of math as science started in ancient Greece. Although math was applied earlier in other ancient civilizations, philosophers like Tales were the first that studied mathematical abstractions like shapes without asking for the practical purpose of that study. Math started to have its very own questions that only could be answered inside Math itself.</p><p>Math became a science. Pythagoreans were a philosophy group that defended that the essence of the universe was numbers.</p><p>But their theory was based on the hypothesis that all numbers could be expressed as a fraction of two integer numbers. It seemed an acceptable supposition until...</p><p>A member of the Pythagorean school discovered some numbers that cannot be expressed that way.</p><p>For example <code>sqrt(2)</code> and <code>sqrt(5)</code>. Those are presumably the first known irrational numbers. So all the Pythagorean theory was reduced to ashes. That was the first big crisis of math.</p><p>What was wrong with the Pythagorean theory? They assumed as true a proposition that is false <strong>inside that theory</strong>. You can prove some numbers cannot be expressed as a fraction using just the theorems, elements, and operations of the theory.</p><p>Greeks realized they needed to change the way math was developed so far. And then Euclid wrote one of the most important books of the history of science: "The Elements". Almost all the geometry we learned until high school was written by Euclid 2500 years earlier. But the best part was the method Euclid used to formulate his geometry. </p><p>"The Elements" showed the first example of an axiomatic theory. An axiomatic theory is built on top of very simple propositions that are assumed as true (axioms). Every other proposition needs to be demonstrated from the axioms by following a set of rules that state how we can go from proposition A to proposition B.</p><p>The process of going from the set of axioms to some proposition A is called demonstration. When we demonstrate proposition A we say that A is a theorem in our theory. Let's see how history continues.</p><p>Euclid built his geometry on top of five axioms. The first four of them seemed pretty simple but the fifth was trickier. It's well known that Euclid himself tried to demonstrate the fifth axiom from the other four.</p><p>More than 2000 years after the first publication of "The Elements", mathematicians were still figuring out how to remove the fifth axiom by demonstrating it. The result of those studies unveiled an astonishing fact: Euclid just defined one of the many possible geometries🤯. If you change the fifth axiom a little bit, you can end up with a perfectly defined (although very crazy) geometry</p><blockquote><p>As a side note, the General Theory of Relativity demonstrated that the geometry of our universe is non-Euclidean (it's one of the new crazy ones).</p></blockquote><p>But this was a big problem! Mathematicians thought the fifth axiom could be demonstrated someday and they built the entire Math building on top of the robust and unique Euclidian geometry. This meant the second deep crisis of Math. We need to rebuild the whole thing again!</p><p>But what do we mean by building the entire Math on top of something?</p><p>It is defining some axioms in a way that any mathematical proposition can be either proved or refuted by a demonstration process. Many of the greatest mathematicians of all time worked hard on that problem. And then, in 1931, a 25 yo man destroyed that intention. Kurt Gdel proved that such a system was impossible to build. He proved that there are true propositions that cannot be proved in some theories. He proved some things cannot be proved! 🤯🤯🤯</p><h2 id="the-theorems">The Theorems</h2><p>First things first. Let's talk about some important concepts.</p><p>We saw what an axiomatic theory is. Well, there are two properties that you'd like to have if you were an axiomatic theory:</p><p>Consistency and Completeness</p><p><strong>Consistency</strong>: A consistent theory is one in which a proposition can be either true or false but not both. In other words, a theory without contradictions. <strong>Inconsistent</strong> theories are useless because you can prove anything from them... Yeah, <em>anything</em>. There's a funny story of Bertrand Russell proving that if 2+2=5 then he was the Pope😆</p><p><strong>Completeness</strong>: A complete theory is one in which all the <em>true</em> propositions are provable inside the theory. The doctoral thesis of Gdel was the demonstration of the completeness of the first-order logic (with 23 yo).</p><p>Now we can continue with the history.</p><p>So, mathematicians were trying to build math on top of other ground different from geometry. They picked number theory (arithmetic, natural numbers) as the new foundations. The main reason: it was axiomatized some years before. To give you an idea of the magnitude of Gdel discovery I'm going to mention some of the mathematicians trying to rebuild math:</p><ul><li>David Hilbert💪</li><li>Bertrand Russel💥</li><li>Ackerman</li><li>John Von Neuman🔥😱🤯💫</li></ul><p>They were trying to prove that the number theory was both Consistent and Complete. That way Math would be safe. The entire Math would be contradictions free and everything could be proven. It seemed to be a matter of time before the proof arrived. Actually, some sub-theories of arithmetic were proven to be both consistent and complete. Gdel himself was working on that but he realized this: </p><p><strong>Theorem 1</strong>: About incompleteness.</p><blockquote><p>For any axiomatic theory <em>that includes a certain part of arithmetic</em>, if it is consistent then, it is incomplete</p></blockquote><p>This means that all theories that include the number theory, contain true propositions that we'll never be able to prove inside that theory!</p><p>All the work of some of the greatest mathematicians of all times was in vain. Jon Von Neuman never worked again in Logic. </p><p>But for those who have some hope in their hearts. I remind you that there are two theorems.</p><p><strong>Theorem 2</strong>: About consistency.</p><blockquote><p>For any consistent theory <em>that contains a certain part of arithmetic</em> the consistency of the theory is not provable.</p></blockquote><p>Precisely one of those true but not provable propositions is the consistency of the theory itself! So, 0 out of 2. No consistency and no completeness. Math can't be built that way. We have to live with that. There are true propositions out there we'll never prove😔. End of story.</p><h3 id="misconceptions">Misconceptions</h3><p>Now, let's talk about some misconceptions generated from the theorems. First I'd like you to note that both theorems say "with a certain amount of arithmetic". </p><p>We will be talking about that amount in the next section. For now, just suppose a theory containing the arithmetic. </p><p>Misconception number one: </p><blockquote><p>Gdel said: for any sufficiently complex theory if it is consistent, then it is incomplete</p></blockquote><p> There is this idea of anything more complex than number theory has the conditions to apply Gdel's incompleteness theorem. Real numbers is a complete theory. And real numbers are at least as complex as natural numbers. It is not about complexity. It is about how natural numbers are defined. That definition has the "poison".</p><p>Misconception number two:</p><blockquote><p>The truth is unreachable for scientists</p></blockquote><p> Ok, some true propositions can't be proven in some theories. But maybe there are other alternative theories. Furthermore, experiments and observations are other methods to discover the truth about our universe.</p><p>Misconception number three:</p><blockquote><p>There is no philosophic system that can explain the universe</p></blockquote><p> The explanation of the universe doesn't have to do with natural numbers necessarily. And Gdel's theorems don't apply when there is no arithmetic in the theory.</p><p>Of course, there are lots more misconceptions about Gdel's results. But I'll stop here🥵. What bout some demonstrations?</p><h2 id="sketch-of-the-demonstrations">Sketch of the demonstrations</h2><p>Let's try to understand how Gdel was able to prove that there are not provable propositions and let's do it as smoothly as we can🙄.</p><p>By the end, we'll have proved one of the most mind-blowing results ever.</p><p>From now we denote with T an axiomatic theory that contains the number theory (theory of natural numbers).</p><p>The first Gdel's theorem states that: If T is consistent, then it is incomplete.</p><p>It means that there are true propositions in T that can't be proven.</p><p>Think about this proposition: "This proposition is not a theorem of T", or equivalently, "This proposition is not provable in T".</p><p>If the proposition was true, then it was a true but unprovable proposition in T. Problem solved. End of the thread. WAIT </p><p>Well, there is a little problem with that last proposition. It talks about the theory T, but it is not a proposition <em>in</em> T. There's a difference between talking about something and being part of it. Let's see how Goedel did it.</p><p>He created a code for every proposition and proof in T, in a way that every one of those propositions and proofs had its own and unique natural number that identifies it. That way, we can talk about the theory from the language of numbers. That's what we call Gdel numeration. But remember that T contains the number theory. And that's the fact Gdel took advantage of.</p><p>Expressing propositions with numbers is a way to talk about T from within T itself. But how?</p><p>By saying: "N is not the code of any theorem in T", we are talking about T. But being the code of a theorem in T is an arithmetic property, and T contains the arithmetic. </p><p>So now we can state the proposition: "The code of this proposition is not the code of any theorem in T". </p><p>But that previous proposition can't still be formulated in T. That's a non-valid syntax. A proposition can't talk about itself in that way.</p><p>We need to achieve that in a more subtle way.</p><p>For that, we'll use the method proposed by Quine that is called "quinning". Let's see the following statement:</p><p>"yields a proposition with property P when appended to its own quotation." yields a proposition with property P when appended to its own quotation.</p><p>We can substitute the quotation for any other sentence. But when using the same sentence the statement starts to talk about itself!</p><p>Gdel proposed another method but it is trickier.</p><p>Let's denote the previous sentence with th letter G. So, if G is true then G has the property P <em>and</em> if G has the property P, then G is true. Let's do the last twist!</p><p>Let's make P = "its code is not the code of any theorem in T". Now, if G is false, then it is not provable, and that makes P true, and that makes G also true. So, T would be inconsistent. If T is consistent, then G is true, then P is also true, and G is not provable!</p><p>We did it!</p><p>G is true but not provable in T!!!</p><p>What about the second theorem?</p><p>"The consistency of T is not provable in T"</p><p>That's a "direct" result from the previous demonstration!</p><p>If T is consistent, then G is true. And that's a demonstration of G, and G is not provable!</p><p>What we know is that if T is consistent then G is true but not provable. But (and precisely because of the previous sentence), the consistency of T is not provable either.</p><p>And that's it. We proved the Goedel theorems!</p><p>Of course, these demonstrations are not too accurate but they comprise the main ideas behind those mind-blowing results.</p><p>So, this is the end of the series😢. Gdel's theorems started a revolution. A revolution that resulted in the birth of Computer Science. But that's another history. Maybe I'll write my own version of that history in the form of threads in the future😉.</p>]]><![CDATA[<p><em>This is a collection of three threads published on Twitter, about Goedel's Theorems.</em></p><h2 id="introduction">Introduction</h2><p>Today I want to start a potentially little series of threads about one of my biggest passions: Gdel's theorems. One of the greatest discoveries of all time that I'm sure will blow your mind! I'll structure the series as follows.</p><p>First: The history that preceded the theorems</p><p>Then: The theorems. Explanation and some common misconceptions</p><p>And finally: Shallow overview of the demonstrations. </p><p>Let's start with the history that preceded Gdel's results. It is pretty much the history of math so let's see if we can comprise it in a thread. </p><h2 id="the-history">The history</h2><p>The study of math as science started in ancient Greece. Although math was applied earlier in other ancient civilizations, philosophers like Tales were the first that studied mathematical abstractions like shapes without asking for the practical purpose of that study. Math started to have its very own questions that only could be answered inside Math itself.</p><p>Math became a science. Pythagoreans were a philosophy group that defended that the essence of the universe was numbers.</p><p>But their theory was based on the hypothesis that all numbers could be expressed as a fraction of two integer numbers. It seemed an acceptable supposition until...</p><p>A member of the Pythagorean school discovered some numbers that cannot be expressed that way.</p><p>For example <code>sqrt(2)</code> and <code>sqrt(5)</code>. Those are presumably the first known irrational numbers. So all the Pythagorean theory was reduced to ashes. That was the first big crisis of math.</p><p>What was wrong with the Pythagorean theory? They assumed as true a proposition that is false <strong>inside that theory</strong>. You can prove some numbers cannot be expressed as a fraction using just the theorems, elements, and operations of the theory.</p><p>Greeks realized they needed to change the way math was developed so far. And then Euclid wrote one of the most important books of the history of science: "The Elements". Almost all the geometry we learned until high school was written by Euclid 2500 years earlier. But the best part was the method Euclid used to formulate his geometry. </p><p>"The Elements" showed the first example of an axiomatic theory. An axiomatic theory is built on top of very simple propositions that are assumed as true (axioms). Every other proposition needs to be demonstrated from the axioms by following a set of rules that state how we can go from proposition A to proposition B.</p><p>The process of going from the set of axioms to some proposition A is called demonstration. When we demonstrate proposition A we say that A is a theorem in our theory. Let's see how history continues.</p><p>Euclid built his geometry on top of five axioms. The first four of them seemed pretty simple but the fifth was trickier. It's well known that Euclid himself tried to demonstrate the fifth axiom from the other four.</p><p>More than 2000 years after the first publication of "The Elements", mathematicians were still figuring out how to remove the fifth axiom by demonstrating it. The result of those studies unveiled an astonishing fact: Euclid just defined one of the many possible geometries🤯. If you change the fifth axiom a little bit, you can end up with a perfectly defined (although very crazy) geometry</p><blockquote><p>As a side note, the General Theory of Relativity demonstrated that the geometry of our universe is non-Euclidean (it's one of the new crazy ones).</p></blockquote><p>But this was a big problem! Mathematicians thought the fifth axiom could be demonstrated someday and they built the entire Math building on top of the robust and unique Euclidian geometry. This meant the second deep crisis of Math. We need to rebuild the whole thing again!</p><p>But what do we mean by building the entire Math on top of something?</p><p>It is defining some axioms in a way that any mathematical proposition can be either proved or refuted by a demonstration process. Many of the greatest mathematicians of all time worked hard on that problem. And then, in 1931, a 25 yo man destroyed that intention. Kurt Gdel proved that such a system was impossible to build. He proved that there are true propositions that cannot be proved in some theories. He proved some things cannot be proved! 🤯🤯🤯</p><h2 id="the-theorems">The Theorems</h2><p>First things first. Let's talk about some important concepts.</p><p>We saw what an axiomatic theory is. Well, there are two properties that you'd like to have if you were an axiomatic theory:</p><p>Consistency and Completeness</p><p><strong>Consistency</strong>: A consistent theory is one in which a proposition can be either true or false but not both. In other words, a theory without contradictions. <strong>Inconsistent</strong> theories are useless because you can prove anything from them... Yeah, <em>anything</em>. There's a funny story of Bertrand Russell proving that if 2+2=5 then he was the Pope😆</p><p><strong>Completeness</strong>: A complete theory is one in which all the <em>true</em> propositions are provable inside the theory. The doctoral thesis of Gdel was the demonstration of the completeness of the first-order logic (with 23 yo).</p><p>Now we can continue with the history.</p><p>So, mathematicians were trying to build math on top of other ground different from geometry. They picked number theory (arithmetic, natural numbers) as the new foundations. The main reason: it was axiomatized some years before. To give you an idea of the magnitude of Gdel discovery I'm going to mention some of the mathematicians trying to rebuild math:</p><ul><li>David Hilbert💪</li><li>Bertrand Russel💥</li><li>Ackerman</li><li>John Von Neuman🔥😱🤯💫</li></ul><p>They were trying to prove that the number theory was both Consistent and Complete. That way Math would be safe. The entire Math would be contradictions free and everything could be proven. It seemed to be a matter of time before the proof arrived. Actually, some sub-theories of arithmetic were proven to be both consistent and complete. Gdel himself was working on that but he realized this: </p><p><strong>Theorem 1</strong>: About incompleteness.</p><blockquote><p>For any axiomatic theory <em>that includes a certain part of arithmetic</em>, if it is consistent then, it is incomplete</p></blockquote><p>This means that all theories that include the number theory, contain true propositions that we'll never be able to prove inside that theory!</p><p>All the work of some of the greatest mathematicians of all times was in vain. Jon Von Neuman never worked again in Logic. </p><p>But for those who have some hope in their hearts. I remind you that there are two theorems.</p><p><strong>Theorem 2</strong>: About consistency.</p><blockquote><p>For any consistent theory <em>that contains a certain part of arithmetic</em> the consistency of the theory is not provable.</p></blockquote><p>Precisely one of those true but not provable propositions is the consistency of the theory itself! So, 0 out of 2. No consistency and no completeness. Math can't be built that way. We have to live with that. There are true propositions out there we'll never prove😔. End of story.</p><h3 id="misconceptions">Misconceptions</h3><p>Now, let's talk about some misconceptions generated from the theorems. First I'd like you to note that both theorems say "with a certain amount of arithmetic". </p><p>We will be talking about that amount in the next section. For now, just suppose a theory containing the arithmetic. </p><p>Misconception number one: </p><blockquote><p>Gdel said: for any sufficiently complex theory if it is consistent, then it is incomplete</p></blockquote><p> There is this idea of anything more complex than number theory has the conditions to apply Gdel's incompleteness theorem. Real numbers is a complete theory. And real numbers are at least as complex as natural numbers. It is not about complexity. It is about how natural numbers are defined. That definition has the "poison".</p><p>Misconception number two:</p><blockquote><p>The truth is unreachable for scientists</p></blockquote><p> Ok, some true propositions can't be proven in some theories. But maybe there are other alternative theories. Furthermore, experiments and observations are other methods to discover the truth about our universe.</p><p>Misconception number three:</p><blockquote><p>There is no philosophic system that can explain the universe</p></blockquote><p> The explanation of the universe doesn't have to do with natural numbers necessarily. And Gdel's theorems don't apply when there is no arithmetic in the theory.</p><p>Of course, there are lots more misconceptions about Gdel's results. But I'll stop here🥵. What bout some demonstrations?</p><h2 id="sketch-of-the-demonstrations">Sketch of the demonstrations</h2><p>Let's try to understand how Gdel was able to prove that there are not provable propositions and let's do it as smoothly as we can🙄.</p><p>By the end, we'll have proved one of the most mind-blowing results ever.</p><p>From now we denote with T an axiomatic theory that contains the number theory (theory of natural numbers).</p><p>The first Gdel's theorem states that: If T is consistent, then it is incomplete.</p><p>It means that there are true propositions in T that can't be proven.</p><p>Think about this proposition: "This proposition is not a theorem of T", or equivalently, "This proposition is not provable in T".</p><p>If the proposition was true, then it was a true but unprovable proposition in T. Problem solved. End of the thread. WAIT </p><p>Well, there is a little problem with that last proposition. It talks about the theory T, but it is not a proposition <em>in</em> T. There's a difference between talking about something and being part of it. Let's see how Goedel did it.</p><p>He created a code for every proposition and proof in T, in a way that every one of those propositions and proofs had its own and unique natural number that identifies it. That way, we can talk about the theory from the language of numbers. That's what we call Gdel numeration. But remember that T contains the number theory. And that's the fact Gdel took advantage of.</p><p>Expressing propositions with numbers is a way to talk about T from within T itself. But how?</p><p>By saying: "N is not the code of any theorem in T", we are talking about T. But being the code of a theorem in T is an arithmetic property, and T contains the arithmetic. </p><p>So now we can state the proposition: "The code of this proposition is not the code of any theorem in T". </p><p>But that previous proposition can't still be formulated in T. That's a non-valid syntax. A proposition can't talk about itself in that way.</p><p>We need to achieve that in a more subtle way.</p><p>For that, we'll use the method proposed by Quine that is called "quinning". Let's see the following statement:</p><p>"yields a proposition with property P when appended to its own quotation." yields a proposition with property P when appended to its own quotation.</p><p>We can substitute the quotation for any other sentence. But when using the same sentence the statement starts to talk about itself!</p><p>Gdel proposed another method but it is trickier.</p><p>Let's denote the previous sentence with th letter G. So, if G is true then G has the property P <em>and</em> if G has the property P, then G is true. Let's do the last twist!</p><p>Let's make P = "its code is not the code of any theorem in T". Now, if G is false, then it is not provable, and that makes P true, and that makes G also true. So, T would be inconsistent. If T is consistent, then G is true, then P is also true, and G is not provable!</p><p>We did it!</p><p>G is true but not provable in T!!!</p><p>What about the second theorem?</p><p>"The consistency of T is not provable in T"</p><p>That's a "direct" result from the previous demonstration!</p><p>If T is consistent, then G is true. And that's a demonstration of G, and G is not provable!</p><p>What we know is that if T is consistent then G is true but not provable. But (and precisely because of the previous sentence), the consistency of T is not provable either.</p><p>And that's it. We proved the Goedel theorems!</p><p>Of course, these demonstrations are not too accurate but they comprise the main ideas behind those mind-blowing results.</p><p>So, this is the end of the series😢. Gdel's theorems started a revolution. A revolution that resulted in the birth of Computer Science. But that's another history. Maybe I'll write my own version of that history in the form of threads in the future😉.</p>]]><![CDATA[Predictive Analytics in plain English]]>https://blog.jayjaydev.com/predictive-analytics-in-plain-englishhttps://blog.jayjaydev.com/predictive-analytics-in-plain-englishTue, 05 Jan 2021 15:05:04 GMT<![CDATA[<p>The field of Predictive Analytics is everywhere these days. We can hear about it in several and different areas like medicine, businesses, industries, and so on. But what exactly Predictive Analytics is? How can some algorithms predict the future, and what is the process to effectively apply Predictive Analytics? This article has the answers to these questions and other insights about this topic.</p><p>In the next section, well be talking about what Predictive Analytics is and what are the fields it comprises. Then, we show different Predictive Analytics approaches. Finally, we describe the stages that usually are part of the Predictive Analytics pipeline.</p><p>So, what do we mean by Predictive Analytics?</p><h2 id="what-exactly-predictive-analytics-is">What exactly Predictive Analytics is?</h2><p>Predictive Analytics is an application of many techniques from Statistics and Artificial Intelligence, especially Machine Learning. It uses current and past facts to predict what to expect from the future.</p><p>For example, large industries can use Predictive Analytics to predict what would be the demand for some products and then adjust the production. Insurance companies use Predictive Analytics to determine whether an accident can occur with high probability and then charge more money.</p><p>Those analyses are carried on by statistical methods and Machine Learning techniques like Data Mining, Hypothesis Tests, and Predictive Modeling. The boom of these techniques is also motivated by technological advances that allow us to make more computations in much less time.</p><p>So, the idea is to accurately predict the future based on current and past data. That is why data has become one of the most valuable assets these days. Big companies like Google and Facebook are big mostly because of all the data they own. There is nothing to do without data. Data is the fuel of the Predictive Analytics engine.</p><p>If we have data, there are many ways to go, depending on what we want to do. The next section is about those different ways.</p><h2 id="types-of-predictive-analytics">Types of Predictive Analytics</h2><p>Usually, we refer to Predictive Analytics as a set of techniques to calculate the likelihood of a future event. That can suggest that Predictive Analytics is all about predictive models that calculate some probabilities and that kind of stuff. While that is often the way to go, other approaches can be considered part of the Predictive Analytics spectrum. Furthermore, for some difficult tasks, it is possible to combine two or more approaches.</p><h3 id="predictive-models">Predictive Models</h3><p>As stated above, this is the most frequent approach. We have many samples of past data, that are called training samples. For example, if we are trying to predict whether a certain transaction could be a fraud, we should have samples of past transactions. Every sample might have a date, a cardholder, the amount of money, the credit card number, and whether that specific transaction was fraudulent or not.</p><p>With those samples, we can train our model and it could be able to predict if a future transaction will be a fraud by knowing the rest of the attributes (cardholder, credit card number, etc). That prediction can be done by calculating a probability. Calculating how likely is a certain transaction to be fraudulent.</p><p>We can use predictive models to make other kinds of predictions. For example, instead of calculating a probability, we can calculate earnings for a future month, based on previous months. Our samples would be now the previous months, every month with attributes like the amount of money invested in advertising, the cost of production, and the earnings. Then we can train a model to predict future earnings. This can help businesses to make important decisions.</p><p>Examples of predictive models are Linear and Logistic Regression, Support Vector Machines, and Neural Networks.</p><h3 id="descriptive-models">Descriptive Models</h3><p>Sometimes we just want to discover some patterns or relationships between the samples. For example, a streaming company or an online retailer wants to recommend similar things to similar people. Then, we might need to identify some categories for customers. There will be customers that like technology, others that can have a low budget to spend, etc.</p><p>We can also use descriptive models to help us to understand better the data. And it could be just a previous step after which we can apply predictive or decision modeling.</p><p>Examples of Descriptive Models are Clustering and Principal Component Analysis.</p><h3 id="decision-models">Decision Models</h3><p>The last approach well be talking about is decision modeling. This one is used when we want to find the relation between all the elements that influence a decision.</p><p>For example, in medicine, we want to build methods that help a doctor to diagnose a disease. But having a black-box that receives the symptoms and prints yes or no, or just pneumonia or heart attack is not useful. We need some explanation about why the model got that result. This way, the doctor can determine whether to trust the algorithm and it can also help the doctor to see some other insights about the patient.</p><p>In this approach, we find relationships between the attributes of the samples and make decisions. Examples of decision models are Decision Trees, Random Forests, and Bayesian Networks.</p><p>After describing the different ways we can encounter Predictive Analytics, lets see a common pipeline of a Predictive Analytics task.</p><h2 id="predictive-analytics-pipeline">Predictive Analytics pipeline</h2><p>Although there are many ways to approach Predictive Analytics depending on the task to solve, we can split the process into some stages that are present in almost all solutions.</p><h3 id="1-data-gathering-cleaning-and-processing">1. Data gathering, cleaning, and processing</h3><p>This is the first and more difficult stage. It usually takes about 75-80% of the total time spent.</p><p>Data is not always easily available. Sometimes you might need to scrap websites and to process thousands of web pages to obtain that precious data. Other methods to obtain data could be even more time-consuming. For example, posting a poll, or a survey and waiting for people to fill them.</p><p>Cleaning data is about getting rid of samples and attributes that arent useful. Maybe we have samples with some missing attributes, or with values that are not correct. This step is extremely important because data often contain noise that can lead to terrible predictions.</p><p>Having clean samples doesnt mean we have the data we need. We must always process the data. Sometimes we need the numerical values to be in a range between 0 and 1, or the literal values to be numerical values, and so on.</p><p>In this processing step, we can do what is called feature engineering. Feature engineering is a group of tricks to make our data even more suitable for the problem. We can make transformations between some attributes and obtain others that are more descriptive or that highlight a pattern.</p><h3 id="2-hypothesis-testing-and-data-visualization">2. Hypothesis Testing and data visualization</h3><p>After processing our data we can state some hypotheses about that data, and how the future can be predicted from it. For example, we can assume that there is a linear relationship between some attributes of the samples. We can also assume that the behavior of an attribute is independent of the behavior of another attribute.</p><p>Those assumptions can be tested. Sometimes we can test whether they hold or not. This formal proof can be done via Hypothesis Testing, which is a powerful method to formally prove the validity of an assumption.</p><p>By visualizing some features of the samples we can also determine whether our assumptions seem to hold or not. Data visualization doesnt give us formal proof but it is extremely helpful in practice. We can visualize some patterns or determine that some assumptions dont seem to hold. Hence, data visualization is useful for both, understanding the data, and asserting whether a hypothesis seems to be correct.</p><h3 id="3-modeling-training-and-validating">3. Modeling, training, and validating</h3><p>Once we have tested our hypothesis about the data, we are ready to build a model to predict the future.</p><p>Our model can be any of the examples mentioned in the previous section or others that were not mentioned. The model can even be a combination of several approaches and models.</p><p>The training process varies depending on the model and the approach we are following. But after the process is completed we should end with a result that should be validated. That means that we need to verify if our model is really solving the problem we want to solve.</p><p>The validation is commonly carried on with another set of samples that are not used in the training process. This way we can determine if our model can solve the task or if it cheated and just perform well with the training data. The validation process can also vary a lot depending on the approach we decided to follow.</p><p>After training and validating, we can obtain two results. If our model seems to perform well, then we can proceed to the next stage. But if our model is not performing as we expected, then we need to think about the cause of that underperformance. Maybe our hypotheses are wrong and we didnt make a correct test. Or it could be some problems with the data (including not enough data). In any case, we need to return to a previous step and repeat part or all the process.</p><h3 id="4-productizing">4. Productizing</h3><p>By productizing, we mean to make the solution available to be used. It could be an online service like the recommendation system of Amazon or Netflix. It could be a desktop application to help a company in the decision-making process. The idea is to make the solution available to the end-user.</p><p>Cloud services have become a suitable and useful environment to host Predictive Analytics products in recent years.</p><p>There is one last stage. After the release of our product, a lot of new data will appear. After some time, the world will change significantly, and our model might get outdated. So, collecting new fresh data and retraining our model is the last step of this process that we can view now as a cycle.</p><h2 id="conclusion">Conclusion</h2><p>In this article, we explained what Predictive Analytics is, and how it is used.</p><p>Every single successful company these days uses Predictive Analytics. Actually, they are successful in part because of using Predictive Analytics.</p><p>This is a weapon that combines the power of Computer Science and Mathematics to predict the future and help to make that future better or worse for many people. By understanding the methods and processes involved in Predictive Analytics, we can help to make that future better for everyone.</p>]]><![CDATA[<p>The field of Predictive Analytics is everywhere these days. We can hear about it in several and different areas like medicine, businesses, industries, and so on. But what exactly Predictive Analytics is? How can some algorithms predict the future, and what is the process to effectively apply Predictive Analytics? This article has the answers to these questions and other insights about this topic.</p><p>In the next section, well be talking about what Predictive Analytics is and what are the fields it comprises. Then, we show different Predictive Analytics approaches. Finally, we describe the stages that usually are part of the Predictive Analytics pipeline.</p><p>So, what do we mean by Predictive Analytics?</p><h2 id="what-exactly-predictive-analytics-is">What exactly Predictive Analytics is?</h2><p>Predictive Analytics is an application of many techniques from Statistics and Artificial Intelligence, especially Machine Learning. It uses current and past facts to predict what to expect from the future.</p><p>For example, large industries can use Predictive Analytics to predict what would be the demand for some products and then adjust the production. Insurance companies use Predictive Analytics to determine whether an accident can occur with high probability and then charge more money.</p><p>Those analyses are carried on by statistical methods and Machine Learning techniques like Data Mining, Hypothesis Tests, and Predictive Modeling. The boom of these techniques is also motivated by technological advances that allow us to make more computations in much less time.</p><p>So, the idea is to accurately predict the future based on current and past data. That is why data has become one of the most valuable assets these days. Big companies like Google and Facebook are big mostly because of all the data they own. There is nothing to do without data. Data is the fuel of the Predictive Analytics engine.</p><p>If we have data, there are many ways to go, depending on what we want to do. The next section is about those different ways.</p><h2 id="types-of-predictive-analytics">Types of Predictive Analytics</h2><p>Usually, we refer to Predictive Analytics as a set of techniques to calculate the likelihood of a future event. That can suggest that Predictive Analytics is all about predictive models that calculate some probabilities and that kind of stuff. While that is often the way to go, other approaches can be considered part of the Predictive Analytics spectrum. Furthermore, for some difficult tasks, it is possible to combine two or more approaches.</p><h3 id="predictive-models">Predictive Models</h3><p>As stated above, this is the most frequent approach. We have many samples of past data, that are called training samples. For example, if we are trying to predict whether a certain transaction could be a fraud, we should have samples of past transactions. Every sample might have a date, a cardholder, the amount of money, the credit card number, and whether that specific transaction was fraudulent or not.</p><p>With those samples, we can train our model and it could be able to predict if a future transaction will be a fraud by knowing the rest of the attributes (cardholder, credit card number, etc). That prediction can be done by calculating a probability. Calculating how likely is a certain transaction to be fraudulent.</p><p>We can use predictive models to make other kinds of predictions. For example, instead of calculating a probability, we can calculate earnings for a future month, based on previous months. Our samples would be now the previous months, every month with attributes like the amount of money invested in advertising, the cost of production, and the earnings. Then we can train a model to predict future earnings. This can help businesses to make important decisions.</p><p>Examples of predictive models are Linear and Logistic Regression, Support Vector Machines, and Neural Networks.</p><h3 id="descriptive-models">Descriptive Models</h3><p>Sometimes we just want to discover some patterns or relationships between the samples. For example, a streaming company or an online retailer wants to recommend similar things to similar people. Then, we might need to identify some categories for customers. There will be customers that like technology, others that can have a low budget to spend, etc.</p><p>We can also use descriptive models to help us to understand better the data. And it could be just a previous step after which we can apply predictive or decision modeling.</p><p>Examples of Descriptive Models are Clustering and Principal Component Analysis.</p><h3 id="decision-models">Decision Models</h3><p>The last approach well be talking about is decision modeling. This one is used when we want to find the relation between all the elements that influence a decision.</p><p>For example, in medicine, we want to build methods that help a doctor to diagnose a disease. But having a black-box that receives the symptoms and prints yes or no, or just pneumonia or heart attack is not useful. We need some explanation about why the model got that result. This way, the doctor can determine whether to trust the algorithm and it can also help the doctor to see some other insights about the patient.</p><p>In this approach, we find relationships between the attributes of the samples and make decisions. Examples of decision models are Decision Trees, Random Forests, and Bayesian Networks.</p><p>After describing the different ways we can encounter Predictive Analytics, lets see a common pipeline of a Predictive Analytics task.</p><h2 id="predictive-analytics-pipeline">Predictive Analytics pipeline</h2><p>Although there are many ways to approach Predictive Analytics depending on the task to solve, we can split the process into some stages that are present in almost all solutions.</p><h3 id="1-data-gathering-cleaning-and-processing">1. Data gathering, cleaning, and processing</h3><p>This is the first and more difficult stage. It usually takes about 75-80% of the total time spent.</p><p>Data is not always easily available. Sometimes you might need to scrap websites and to process thousands of web pages to obtain that precious data. Other methods to obtain data could be even more time-consuming. For example, posting a poll, or a survey and waiting for people to fill them.</p><p>Cleaning data is about getting rid of samples and attributes that arent useful. Maybe we have samples with some missing attributes, or with values that are not correct. This step is extremely important because data often contain noise that can lead to terrible predictions.</p><p>Having clean samples doesnt mean we have the data we need. We must always process the data. Sometimes we need the numerical values to be in a range between 0 and 1, or the literal values to be numerical values, and so on.</p><p>In this processing step, we can do what is called feature engineering. Feature engineering is a group of tricks to make our data even more suitable for the problem. We can make transformations between some attributes and obtain others that are more descriptive or that highlight a pattern.</p><h3 id="2-hypothesis-testing-and-data-visualization">2. Hypothesis Testing and data visualization</h3><p>After processing our data we can state some hypotheses about that data, and how the future can be predicted from it. For example, we can assume that there is a linear relationship between some attributes of the samples. We can also assume that the behavior of an attribute is independent of the behavior of another attribute.</p><p>Those assumptions can be tested. Sometimes we can test whether they hold or not. This formal proof can be done via Hypothesis Testing, which is a powerful method to formally prove the validity of an assumption.</p><p>By visualizing some features of the samples we can also determine whether our assumptions seem to hold or not. Data visualization doesnt give us formal proof but it is extremely helpful in practice. We can visualize some patterns or determine that some assumptions dont seem to hold. Hence, data visualization is useful for both, understanding the data, and asserting whether a hypothesis seems to be correct.</p><h3 id="3-modeling-training-and-validating">3. Modeling, training, and validating</h3><p>Once we have tested our hypothesis about the data, we are ready to build a model to predict the future.</p><p>Our model can be any of the examples mentioned in the previous section or others that were not mentioned. The model can even be a combination of several approaches and models.</p><p>The training process varies depending on the model and the approach we are following. But after the process is completed we should end with a result that should be validated. That means that we need to verify if our model is really solving the problem we want to solve.</p><p>The validation is commonly carried on with another set of samples that are not used in the training process. This way we can determine if our model can solve the task or if it cheated and just perform well with the training data. The validation process can also vary a lot depending on the approach we decided to follow.</p><p>After training and validating, we can obtain two results. If our model seems to perform well, then we can proceed to the next stage. But if our model is not performing as we expected, then we need to think about the cause of that underperformance. Maybe our hypotheses are wrong and we didnt make a correct test. Or it could be some problems with the data (including not enough data). In any case, we need to return to a previous step and repeat part or all the process.</p><h3 id="4-productizing">4. Productizing</h3><p>By productizing, we mean to make the solution available to be used. It could be an online service like the recommendation system of Amazon or Netflix. It could be a desktop application to help a company in the decision-making process. The idea is to make the solution available to the end-user.</p><p>Cloud services have become a suitable and useful environment to host Predictive Analytics products in recent years.</p><p>There is one last stage. After the release of our product, a lot of new data will appear. After some time, the world will change significantly, and our model might get outdated. So, collecting new fresh data and retraining our model is the last step of this process that we can view now as a cycle.</p><h2 id="conclusion">Conclusion</h2><p>In this article, we explained what Predictive Analytics is, and how it is used.</p><p>Every single successful company these days uses Predictive Analytics. Actually, they are successful in part because of using Predictive Analytics.</p><p>This is a weapon that combines the power of Computer Science and Mathematics to predict the future and help to make that future better or worse for many people. By understanding the methods and processes involved in Predictive Analytics, we can help to make that future better for everyone.</p>]]><![CDATA[Greedy strategy can be useful even when it doesn't work]]>https://blog.jayjaydev.com/greedy-strategy-can-be-useful-even-when-it-doesnt-workhttps://blog.jayjaydev.com/greedy-strategy-can-be-useful-even-when-it-doesnt-workSat, 28 Nov 2020 18:55:16 GMT<![CDATA[<p>Greedy algorithms try to find the optimal solution by taking the best available choice at every step. For example, you can greedily approach your life. You can always take the path that maximizes your happiness today. But that doesn't mean you'll be happier tomorrow.</p><p>Similarly, there are problems for which greedy algorithms don't yield the best solution. Actually, they might yield the worst possible solution. But there are other cases in which we can obtain a solution that is good enough by using a greedy strategy.</p><p>In this article, I'll write about greedy algorithms and the use of this strategy even when it doesn't guarantee to find an optimal solution.</p><p>The next section is an introduction to greedy algorithms and well-known problems that are solvable using this strategy. Then I'll talk about problems in which the greedy strategy is a really bad option and finally, I show you an example of a good approximation through a greedy algorithm.</p><blockquote><p><strong>Note</strong>: Most of the algorithms and problems I discuss in this article include graphs. It would be good if you are familiarized with graphs to get the most out of this post.</p></blockquote><h2 id="greedy-algorithms">Greedy algorithms</h2><p>Greedy algorithms always choose the best available option. In general, they are computationally cheaper than other families of algorithms like dynamic programming, or brute force. That's because they don't explore the solution space too much. And, for the same reason, they don't find the best solution to a lot of problems.</p><p>But there are lots of problems that are solvable with a greedy strategy, and that strategy is precisely the best way to go.</p><p>One of the most popular greedy algorithms is Dijkstra's algorithm to find the path with the minimum cost from one vertex to the others in a graph. This algorithm finds such a path by always going to the nearest vertex. That's why we say it is a greedy algorithm.</p><p>This is a pseudocode of the algorithm. I denote with <code>G</code> the graph and with <code>s</code> the source node. </p><pre><code class="lang-pseudocode">Dijkstra(G, s): distances <- list of length equal to the number of nodes of the graph, initially it has all its elements equal to infinite distances[s] = 0 queue = the set of vertices of G while queue is not empty: u <- vertex in queue with min distances[u] remove u from queue for each neighbor v of u: temp = distances[u] + value(u,v) if temp < distances[v]: distances[v] = temp return distances</code></pre><p>After running the previous algorithm we get a list <code>distances</code> such that <code>distances[u]</code> is the minimum cost to go from node <code>s</code> to node <code>u</code>.</p><p>This algorithm is guaranteed to work only if the graph hasn't edges with negative costs. A negative cost in an edge can make the greedy strategy to choose a path that is not optimum.</p><p>Another example that is used to introduce the concepts of the greedy strategy is the Fractional Knapsack.</p><p>In this problem, we have a collection of items. Each item has a weight <code>Wi</code> greater than zero, and a profit <code>Pi</code> also greater than zero. We have a knapsack with a capacity <code>W</code> and we want to fill it in such a way that we get the maximum profit. Of course, we cannot exceed the capacity of the knapsack.</p><p>In the fractional version of the knapsack problem, we can take either the entire object or only a fraction of it. When taking a fraction <code>0 <= X <= 1</code> of the i-th object, we obtain a profit equal to <code>X*Pi</code> and we need to add <code>X*Wi</code> to the bag. We can solve this problem by using a greedy strategy. I won't discuss the solution here. If you don't know it I recommend you try to solve it by yourself and then look for the solution online.</p><p>The number of problems that we can solve by using greedy algorithms is huge. But the number of problems that we cannot solve this way is even bigger. The next section is about the latter ones.</p><h2 id="when-being-greedy-is-the-worst">When being greedy is the worst</h2><p>In the previous section, we saw two examples of problems that are solvable using a greedy strategy. This is great because those are pretty fast algorithms.</p><p>But, as I said, Dijkstra's algorithm doesn't work in graphs with negative edges. And the problem is even bigger. I can always build a graph with negative edges in a way that Dijkstra's solution was as bad as I wanted! Consider the following example that was extracted from <a target="_blank" href="https://stackoverflow.com/questions/6799172/negative-weights-using-dijkstras-algorithm/6799344#6799344">Stackoverflow</a></p><p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1606581688656/MrsI2Usdb.png" alt="rmowk.png" /></p><p>Dijkstra's algorithm fails to find the distance between <code>A</code> and <code>C</code>. It finds <code>d(A, C) = 0</code> when it should be -200. And if we decrease the value of the edge <code>D -> B</code>, we'll obtain a distance that we'll be even farther from the actual minimum distance.</p><p>Similarly, when we can't break objects in the knapsack problem (the 0-1 Knapsack Problem), the solution that we obtain when using a greedy strategy can be as bad as we want. We always can build an input to the problem that makes the greedy algorithm to fail badly.</p><p>Another example is the Travelling Salesman Problem (TSP). Given a list of cities and the distances between each pair of cities, what is the shortest possible route that visits each city exactly once and returns to the origin city?</p><p>We can greedily approach the problem by always going to the nearest possible city. We select any of the cities as the first one and apply that strategy.</p><p>As happened in previous examples, we can always build a disposition of the cities in a way that the greedy strategy finds the worst possible solution.</p><p>In this section, we have seen that a greedy strategy could lead us to disaster. But there are problems in which such an approach can approximate the optimal solution quite well.</p><h2 id="when-being-greedy-is-not-that-bad">When being greedy is not that bad</h2><p>We have seen that greedy strategy is as bad as we want for some problems. This means that we cannot use it to obtain the optimal solution nor even a good approximation to it.</p><p>But there are some examples in which greedy algorithms provide us with very good approximations! In these cases, the greedy approach is very useful because it tends to be cheaper and easier to implement.</p><p>The vertex cover of a graph is the minimum set of vertices such that every edge of the graph has at least one of its endpoints in the set.</p><p>This is a very hard problem. Actually, there isn't any efficient and exact solution for it. But the good news is that we can make a good approximation with a greedy algorithm.</p><p>We select any edge <code><u, v></code> from the graph, and add <code>u</code> and <code>v</code> to the set. Then, we remove all the edges that have <code>u</code> or <code>v</code> as one of their endpoints, and we repeat the previous process while the remaining graph had edges.</p><p>This might be a pseudocode of the previous algorithm.</p><pre><code class="lang-pseudocode">vertexCover(G): VertexCover <- {} // empty set E' <- edges of G while E' is not empty: VertexCover <- VertexCover U {u,v} where <u,v> is in E' E' = E' - {<u, v> U edges incident to u, v} return VertexCover</code></pre><p>As you can see, this is a simple and relatively fast algorithm. But the best part is that the solution will always be less or equal to two times the optimal solution! We'll never obtain a set that is bigger than two times the smaller vertex cover, no matter how the input graph was built.</p><p>I'm not going to include the demonstration of this statement in this post, but you can prove it by noticing that for every edge <code><u, v></code> that we add to the vertex cover, either <code>u</code> or <code>v</code> are in the optimal solution (i.e. in the smaller vertex cover).</p><p>Many computer scientists are working to find more of these approximations. There are more examples, but I'm going to stop here. This is an interesting and very active research field in Computer Science and Applied Mathematics. With these approximations, we can get very good solutions for very hard problems by implementing pretty simple algorithms.</p><h2 id="conclusions">Conclusions</h2><p>In this post, I made a shallow introduction to greedy algorithms. We saw examples of problems that can be solved using the greedy strategy. Then, I talk about some problems for which the greedy strategy is too bad, and finally, we saw an example of a greedy algorithm to get an approximated solution for a hard problem.</p><p>Sometimes we can solve a problem using a greedy approach but it is hard to come up with the right strategy. And demonstrating the correctness of greedy algorithms (for exact or approximated solutions) can be very difficult. So, there are a lot of things we can talk about greedy algorithms!</p><p>If you enjoyed this post and want me to keep this type of content coming let me know by reacting and/or commenting. You can also follow me on Twitter for more CS-related content. </p>]]><![CDATA[<p>Greedy algorithms try to find the optimal solution by taking the best available choice at every step. For example, you can greedily approach your life. You can always take the path that maximizes your happiness today. But that doesn't mean you'll be happier tomorrow.</p><p>Similarly, there are problems for which greedy algorithms don't yield the best solution. Actually, they might yield the worst possible solution. But there are other cases in which we can obtain a solution that is good enough by using a greedy strategy.</p><p>In this article, I'll write about greedy algorithms and the use of this strategy even when it doesn't guarantee to find an optimal solution.</p><p>The next section is an introduction to greedy algorithms and well-known problems that are solvable using this strategy. Then I'll talk about problems in which the greedy strategy is a really bad option and finally, I show you an example of a good approximation through a greedy algorithm.</p><blockquote><p><strong>Note</strong>: Most of the algorithms and problems I discuss in this article include graphs. It would be good if you are familiarized with graphs to get the most out of this post.</p></blockquote><h2 id="greedy-algorithms">Greedy algorithms</h2><p>Greedy algorithms always choose the best available option. In general, they are computationally cheaper than other families of algorithms like dynamic programming, or brute force. That's because they don't explore the solution space too much. And, for the same reason, they don't find the best solution to a lot of problems.</p><p>But there are lots of problems that are solvable with a greedy strategy, and that strategy is precisely the best way to go.</p><p>One of the most popular greedy algorithms is Dijkstra's algorithm to find the path with the minimum cost from one vertex to the others in a graph. This algorithm finds such a path by always going to the nearest vertex. That's why we say it is a greedy algorithm.</p><p>This is a pseudocode of the algorithm. I denote with <code>G</code> the graph and with <code>s</code> the source node. </p><pre><code class="lang-pseudocode">Dijkstra(G, s): distances <- list of length equal to the number of nodes of the graph, initially it has all its elements equal to infinite distances[s] = 0 queue = the set of vertices of G while queue is not empty: u <- vertex in queue with min distances[u] remove u from queue for each neighbor v of u: temp = distances[u] + value(u,v) if temp < distances[v]: distances[v] = temp return distances</code></pre><p>After running the previous algorithm we get a list <code>distances</code> such that <code>distances[u]</code> is the minimum cost to go from node <code>s</code> to node <code>u</code>.</p><p>This algorithm is guaranteed to work only if the graph hasn't edges with negative costs. A negative cost in an edge can make the greedy strategy to choose a path that is not optimum.</p><p>Another example that is used to introduce the concepts of the greedy strategy is the Fractional Knapsack.</p><p>In this problem, we have a collection of items. Each item has a weight <code>Wi</code> greater than zero, and a profit <code>Pi</code> also greater than zero. We have a knapsack with a capacity <code>W</code> and we want to fill it in such a way that we get the maximum profit. Of course, we cannot exceed the capacity of the knapsack.</p><p>In the fractional version of the knapsack problem, we can take either the entire object or only a fraction of it. When taking a fraction <code>0 <= X <= 1</code> of the i-th object, we obtain a profit equal to <code>X*Pi</code> and we need to add <code>X*Wi</code> to the bag. We can solve this problem by using a greedy strategy. I won't discuss the solution here. If you don't know it I recommend you try to solve it by yourself and then look for the solution online.</p><p>The number of problems that we can solve by using greedy algorithms is huge. But the number of problems that we cannot solve this way is even bigger. The next section is about the latter ones.</p><h2 id="when-being-greedy-is-the-worst">When being greedy is the worst</h2><p>In the previous section, we saw two examples of problems that are solvable using a greedy strategy. This is great because those are pretty fast algorithms.</p><p>But, as I said, Dijkstra's algorithm doesn't work in graphs with negative edges. And the problem is even bigger. I can always build a graph with negative edges in a way that Dijkstra's solution was as bad as I wanted! Consider the following example that was extracted from <a target="_blank" href="https://stackoverflow.com/questions/6799172/negative-weights-using-dijkstras-algorithm/6799344#6799344">Stackoverflow</a></p><p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1606581688656/MrsI2Usdb.png" alt="rmowk.png" /></p><p>Dijkstra's algorithm fails to find the distance between <code>A</code> and <code>C</code>. It finds <code>d(A, C) = 0</code> when it should be -200. And if we decrease the value of the edge <code>D -> B</code>, we'll obtain a distance that we'll be even farther from the actual minimum distance.</p><p>Similarly, when we can't break objects in the knapsack problem (the 0-1 Knapsack Problem), the solution that we obtain when using a greedy strategy can be as bad as we want. We always can build an input to the problem that makes the greedy algorithm to fail badly.</p><p>Another example is the Travelling Salesman Problem (TSP). Given a list of cities and the distances between each pair of cities, what is the shortest possible route that visits each city exactly once and returns to the origin city?</p><p>We can greedily approach the problem by always going to the nearest possible city. We select any of the cities as the first one and apply that strategy.</p><p>As happened in previous examples, we can always build a disposition of the cities in a way that the greedy strategy finds the worst possible solution.</p><p>In this section, we have seen that a greedy strategy could lead us to disaster. But there are problems in which such an approach can approximate the optimal solution quite well.</p><h2 id="when-being-greedy-is-not-that-bad">When being greedy is not that bad</h2><p>We have seen that greedy strategy is as bad as we want for some problems. This means that we cannot use it to obtain the optimal solution nor even a good approximation to it.</p><p>But there are some examples in which greedy algorithms provide us with very good approximations! In these cases, the greedy approach is very useful because it tends to be cheaper and easier to implement.</p><p>The vertex cover of a graph is the minimum set of vertices such that every edge of the graph has at least one of its endpoints in the set.</p><p>This is a very hard problem. Actually, there isn't any efficient and exact solution for it. But the good news is that we can make a good approximation with a greedy algorithm.</p><p>We select any edge <code><u, v></code> from the graph, and add <code>u</code> and <code>v</code> to the set. Then, we remove all the edges that have <code>u</code> or <code>v</code> as one of their endpoints, and we repeat the previous process while the remaining graph had edges.</p><p>This might be a pseudocode of the previous algorithm.</p><pre><code class="lang-pseudocode">vertexCover(G): VertexCover <- {} // empty set E' <- edges of G while E' is not empty: VertexCover <- VertexCover U {u,v} where <u,v> is in E' E' = E' - {<u, v> U edges incident to u, v} return VertexCover</code></pre><p>As you can see, this is a simple and relatively fast algorithm. But the best part is that the solution will always be less or equal to two times the optimal solution! We'll never obtain a set that is bigger than two times the smaller vertex cover, no matter how the input graph was built.</p><p>I'm not going to include the demonstration of this statement in this post, but you can prove it by noticing that for every edge <code><u, v></code> that we add to the vertex cover, either <code>u</code> or <code>v</code> are in the optimal solution (i.e. in the smaller vertex cover).</p><p>Many computer scientists are working to find more of these approximations. There are more examples, but I'm going to stop here. This is an interesting and very active research field in Computer Science and Applied Mathematics. With these approximations, we can get very good solutions for very hard problems by implementing pretty simple algorithms.</p><h2 id="conclusions">Conclusions</h2><p>In this post, I made a shallow introduction to greedy algorithms. We saw examples of problems that can be solved using the greedy strategy. Then, I talk about some problems for which the greedy strategy is too bad, and finally, we saw an example of a greedy algorithm to get an approximated solution for a hard problem.</p><p>Sometimes we can solve a problem using a greedy approach but it is hard to come up with the right strategy. And demonstrating the correctness of greedy algorithms (for exact or approximated solutions) can be very difficult. So, there are a lot of things we can talk about greedy algorithms!</p><p>If you enjoyed this post and want me to keep this type of content coming let me know by reacting and/or commenting. You can also follow me on Twitter for more CS-related content. </p>]]><![CDATA[A new language for every new problem]]>https://blog.jayjaydev.com/a-new-language-for-every-new-problemhttps://blog.jayjaydev.com/a-new-language-for-every-new-problemMon, 09 Nov 2020 03:05:40 GMT<![CDATA[<p>What if I tell you that you can always have the language that better fits your problem?</p><p>We are always implementing design patterns and abstractions to adapt our programming language to the domain of the problem. We create the classes <code>Store</code>, <code>Product</code>, and <code>Customer</code>. We create functions to represent the operations of selling, buying, updating the inventory, etc. But all those building blocks are always expressed in terms of a general-purpose language that was not thought to fit our very immediate necessities.</p><p>In this article, I'll talk about the paradigm that is used in the Lisp programming language. With Lisp, we not only create a program but also create a specific language at the same time. Thus, we end up with a Domain-Specific Language (DSL) and a short, elegant, and clean program.</p><h2 id="heading-understanding-the-point">Understanding the point</h2><p>We always create new entities and operators when programming. We implement functions with meaningful names that express part of the domain of our problem. But those functions and classes are created in the same way, with a rigid syntactical structure. No matter how elaborated your abstractions are, you always build them in the same way. For example, creating a class in Python will always be like this:</p><pre><code class="lang-python"><span class="hljs-class"><span class="hljs-keyword">class</span> <<span class="hljs-title">ClassName</span>>(<span class="hljs-params"><ParentClass></span>):</span> <ClassBody></code></pre><p>Maybe you can imagine another more expressive way to create classes in the domain you are working in, but there is no way to change that syntax into a kind of shortcut to create those domain-specific classes. You can solve a lot of problems with inheritance and other mechanisms, but a function always needs to be built with the same syntactical structure, and the same happens with classes.</p><p>You can redefine operators like <code>+</code>, and <code>==</code>. But you can't modify what the <code>class</code> reserved word does. You can't add a new operator to create classes either.</p><p>All the programs you create in Python are written with the same vocabulary. Your coding expertise can make the code to be expressive, but no matter the expertise, that expressiveness will always have a limit. A limit imposed by the syntax of the programming language and its reserved words.</p><p>Most of the developer community is so used to these limitations that not even question them. We learn to code in a specific language, we learn some algorithms and the way we can translate them into that programming language. We get used to building programs in that way.</p><p>But there's another way. I just want to make a little detour right now and then dive into the main topic.</p><h2 id="heading-limitations-of-c-like-macros">Limitations of C-like macros</h2><p>If you have programmed in C/C++, Assembly, or some other language that supports macros (but not the Lisp-like macros), you could think that the previous section problem does not apply to that language.</p><p>There is a big difference between C-like macros and what we're going to talk about in the next section.</p><p>With C-like macros, we can define a pattern and then substitute any occurrence of that pattern in our program for a piece of code, in compilation time. For example, we can do</p><pre><code class="lang-c"><span class="hljs-meta">#<span class="hljs-meta-keyword">define</span> MAX 1000000</span>...<span class="hljs-keyword">int</span> <span class="hljs-built_in">array</span>[MAX];...</code></pre><p>Before actually compiling the program, the C compiler substitutes any occurrence of <code>MAX</code> for <code>1000000</code>. Extending that functionality, we could modify the syntax for <code>for</code> loops and <code>if</code> statements if we wanted. But there's still an important limitation.</p><p>We don't have all the power of the language to define those macros. We just can define some mapping between string patterns and pieces of code. And macros can't make any computation in compilation time. They're just a text substitution, nothing more.</p><p>Let's see the real power of Lisp.</p><h2 id="heading-creating-a-new-language-every-time">Creating a new language every time</h2><p>As I said before, when you code in Lisp (in the right way), you are doing two things. Firstly, create a new language that is specific to your domain, and secondly, code a program in that expressive and very convenient language you've just created.</p><p>Actually, there's no order. You create both the language and the program simultaneously. Both of them evolve during the development process and the final product is a highly readable and clean program.</p><p>You can use another brand-new operator to create classes that represent stores for example:</p><pre><code class="lang-python">(define-store my-store (:capacity <span class="hljs-number">100</span> :budget <span class="hljs-number">100000</span> :description <span class="hljs-string">"My very first store template"</span>))</code></pre><p>That could be equivalent to building a class named <code>my-store</code>, which might implicitly inherit from a <code>store</code> parent class. We are passing a <code>capacity</code>, a <code>budget</code>, and a <code>description</code>. Then, in compilation time, the <code>define-store</code> operator can do all the things you can imagine, maybe it is not even a class definition. The thing is, we are declaring an entity in a very clear and domain-specific way.</p><p>It might be a toy example and it is totally out of context. But the cool thing is that you can imagine any sort of complicated computations running under the hood, and that will be a possible scenario. With enough expertise, you can transform Lisp into the language you need to make the shortest and cleanest program.</p><p>In the above example, the operator <code>define-store</code> is a macro. A macro in Lisp is like a special function. They can receive parameters and they are executed in compilation time, in a step called <em>macroexpansion</em>. Then, the macro call is substituted by its own result in compilation time. For example, we can define the following macro:</p><pre><code class="lang-python">(defmacro hundred () (* <span class="hljs-number">2</span> <span class="hljs-number">50</span>))</code></pre><p>We defined a macro called <code>hundred</code> that receives no parameters. It returns the result of multiplying 2 and 50.</p><blockquote><p>Notice that we write the arithmetic operator first and the operands later. One of the advantages is that we can make <code>(* 2 50 100 200)</code> and it will return the product of all those factors. But of course, you can use some advanced Lisp magic to change this syntax ;-)</p></blockquote><p>Now, what happens if we write</p><pre><code class="lang-python">(+ hundred <span class="hljs-number">30</span>)</code></pre><p>The first step is <em>macroexpansion</em>. The compiler substitutes the macro call for the result of that call and the code becomes:</p><pre><code class="lang-python">(+ <span class="hljs-number">100</span> <span class="hljs-number">30</span>)</code></pre><p>Then, the program is executed and we obtain the result <code>130</code>.</p><p>Notice the difference with C-like macros. There was more than just text substitution, there was computation and then, code substitution.</p><p>If we change the macro definition by adding just one more character (notice the quote):</p><pre><code class="lang-python">(defmacro hundred () <span class="hljs-string">'(* 2 50))</span></code></pre><p>Then the <em>macroexpansion</em> changes a little bit too:</p><pre><code class="lang-python">(+ (* <span class="hljs-number">2</span> <span class="hljs-number">50</span>) <span class="hljs-number">30</span>)</code></pre><p>But the output remains the same. Now the macro returns the code as-is, and there are no further computations in compilation time.</p><h2 id="heading-beyond-macros">Beyond macros</h2><p>Talking deeper about Lisp macros requires to have a basic knowledge of Lisp programming language. In this article, I don't assume any prior knowledge about Lisp. So, the examples I can use are very limited. It wouldn't make any sense to include a complex macro definition. But my point is not to teach you how to define some macros in Lisp but to show you another way of thinking about programming.</p><p>The idea of building your program and the language in which your program is written at the same time is, at least, beautiful. The elegance of the process can only be compared with that of the final result.</p><p>We end up with two products. The desired program, but also a language that we can use totally or just partially in our next journey.</p><p>Just knowing about this philosophy is totally worth it.</p><h2 id="heading-conclusions">Conclusions</h2><p>In this article, I have talked about Lisp's macros.</p><p>They are a powerful feature that allows us to define new Domain-Specific Languages for every problem we face. With them, we can create the cleanest programs and get the language in which that program is written at the same time.</p><p>That immense power makes us approach programming in a very different way.</p><p>This article is part of a series called "Cool things about Lisp". I plan to add more articles to this series in the future. Trust me, there's a lot of more cool things to talk about. Let me know if you like this content and want me to keep writing about it. Leave your impression and/or your comment.</p><p>You can follow me on Twitter. I'm always writing about Computer Science stuff.</p>]]><![CDATA[<p>What if I tell you that you can always have the language that better fits your problem?</p><p>We are always implementing design patterns and abstractions to adapt our programming language to the domain of the problem. We create the classes <code>Store</code>, <code>Product</code>, and <code>Customer</code>. We create functions to represent the operations of selling, buying, updating the inventory, etc. But all those building blocks are always expressed in terms of a general-purpose language that was not thought to fit our very immediate necessities.</p><p>In this article, I'll talk about the paradigm that is used in the Lisp programming language. With Lisp, we not only create a program but also create a specific language at the same time. Thus, we end up with a Domain-Specific Language (DSL) and a short, elegant, and clean program.</p><h2 id="heading-understanding-the-point">Understanding the point</h2><p>We always create new entities and operators when programming. We implement functions with meaningful names that express part of the domain of our problem. But those functions and classes are created in the same way, with a rigid syntactical structure. No matter how elaborated your abstractions are, you always build them in the same way. For example, creating a class in Python will always be like this:</p><pre><code class="lang-python"><span class="hljs-class"><span class="hljs-keyword">class</span> <<span class="hljs-title">ClassName</span>>(<span class="hljs-params"><ParentClass></span>):</span> <ClassBody></code></pre><p>Maybe you can imagine another more expressive way to create classes in the domain you are working in, but there is no way to change that syntax into a kind of shortcut to create those domain-specific classes. You can solve a lot of problems with inheritance and other mechanisms, but a function always needs to be built with the same syntactical structure, and the same happens with classes.</p><p>You can redefine operators like <code>+</code>, and <code>==</code>. But you can't modify what the <code>class</code> reserved word does. You can't add a new operator to create classes either.</p><p>All the programs you create in Python are written with the same vocabulary. Your coding expertise can make the code to be expressive, but no matter the expertise, that expressiveness will always have a limit. A limit imposed by the syntax of the programming language and its reserved words.</p><p>Most of the developer community is so used to these limitations that not even question them. We learn to code in a specific language, we learn some algorithms and the way we can translate them into that programming language. We get used to building programs in that way.</p><p>But there's another way. I just want to make a little detour right now and then dive into the main topic.</p><h2 id="heading-limitations-of-c-like-macros">Limitations of C-like macros</h2><p>If you have programmed in C/C++, Assembly, or some other language that supports macros (but not the Lisp-like macros), you could think that the previous section problem does not apply to that language.</p><p>There is a big difference between C-like macros and what we're going to talk about in the next section.</p><p>With C-like macros, we can define a pattern and then substitute any occurrence of that pattern in our program for a piece of code, in compilation time. For example, we can do</p><pre><code class="lang-c"><span class="hljs-meta">#<span class="hljs-meta-keyword">define</span> MAX 1000000</span>...<span class="hljs-keyword">int</span> <span class="hljs-built_in">array</span>[MAX];...</code></pre><p>Before actually compiling the program, the C compiler substitutes any occurrence of <code>MAX</code> for <code>1000000</code>. Extending that functionality, we could modify the syntax for <code>for</code> loops and <code>if</code> statements if we wanted. But there's still an important limitation.</p><p>We don't have all the power of the language to define those macros. We just can define some mapping between string patterns and pieces of code. And macros can't make any computation in compilation time. They're just a text substitution, nothing more.</p><p>Let's see the real power of Lisp.</p><h2 id="heading-creating-a-new-language-every-time">Creating a new language every time</h2><p>As I said before, when you code in Lisp (in the right way), you are doing two things. Firstly, create a new language that is specific to your domain, and secondly, code a program in that expressive and very convenient language you've just created.</p><p>Actually, there's no order. You create both the language and the program simultaneously. Both of them evolve during the development process and the final product is a highly readable and clean program.</p><p>You can use another brand-new operator to create classes that represent stores for example:</p><pre><code class="lang-python">(define-store my-store (:capacity <span class="hljs-number">100</span> :budget <span class="hljs-number">100000</span> :description <span class="hljs-string">"My very first store template"</span>))</code></pre><p>That could be equivalent to building a class named <code>my-store</code>, which might implicitly inherit from a <code>store</code> parent class. We are passing a <code>capacity</code>, a <code>budget</code>, and a <code>description</code>. Then, in compilation time, the <code>define-store</code> operator can do all the things you can imagine, maybe it is not even a class definition. The thing is, we are declaring an entity in a very clear and domain-specific way.</p><p>It might be a toy example and it is totally out of context. But the cool thing is that you can imagine any sort of complicated computations running under the hood, and that will be a possible scenario. With enough expertise, you can transform Lisp into the language you need to make the shortest and cleanest program.</p><p>In the above example, the operator <code>define-store</code> is a macro. A macro in Lisp is like a special function. They can receive parameters and they are executed in compilation time, in a step called <em>macroexpansion</em>. Then, the macro call is substituted by its own result in compilation time. For example, we can define the following macro:</p><pre><code class="lang-python">(defmacro hundred () (* <span class="hljs-number">2</span> <span class="hljs-number">50</span>))</code></pre><p>We defined a macro called <code>hundred</code> that receives no parameters. It returns the result of multiplying 2 and 50.</p><blockquote><p>Notice that we write the arithmetic operator first and the operands later. One of the advantages is that we can make <code>(* 2 50 100 200)</code> and it will return the product of all those factors. But of course, you can use some advanced Lisp magic to change this syntax ;-)</p></blockquote><p>Now, what happens if we write</p><pre><code class="lang-python">(+ hundred <span class="hljs-number">30</span>)</code></pre><p>The first step is <em>macroexpansion</em>. The compiler substitutes the macro call for the result of that call and the code becomes:</p><pre><code class="lang-python">(+ <span class="hljs-number">100</span> <span class="hljs-number">30</span>)</code></pre><p>Then, the program is executed and we obtain the result <code>130</code>.</p><p>Notice the difference with C-like macros. There was more than just text substitution, there was computation and then, code substitution.</p><p>If we change the macro definition by adding just one more character (notice the quote):</p><pre><code class="lang-python">(defmacro hundred () <span class="hljs-string">'(* 2 50))</span></code></pre><p>Then the <em>macroexpansion</em> changes a little bit too:</p><pre><code class="lang-python">(+ (* <span class="hljs-number">2</span> <span class="hljs-number">50</span>) <span class="hljs-number">30</span>)</code></pre><p>But the output remains the same. Now the macro returns the code as-is, and there are no further computations in compilation time.</p><h2 id="heading-beyond-macros">Beyond macros</h2><p>Talking deeper about Lisp macros requires to have a basic knowledge of Lisp programming language. In this article, I don't assume any prior knowledge about Lisp. So, the examples I can use are very limited. It wouldn't make any sense to include a complex macro definition. But my point is not to teach you how to define some macros in Lisp but to show you another way of thinking about programming.</p><p>The idea of building your program and the language in which your program is written at the same time is, at least, beautiful. The elegance of the process can only be compared with that of the final result.</p><p>We end up with two products. The desired program, but also a language that we can use totally or just partially in our next journey.</p><p>Just knowing about this philosophy is totally worth it.</p><h2 id="heading-conclusions">Conclusions</h2><p>In this article, I have talked about Lisp's macros.</p><p>They are a powerful feature that allows us to define new Domain-Specific Languages for every problem we face. With them, we can create the cleanest programs and get the language in which that program is written at the same time.</p><p>That immense power makes us approach programming in a very different way.</p><p>This article is part of a series called "Cool things about Lisp". I plan to add more articles to this series in the future. Trust me, there's a lot of more cool things to talk about. Let me know if you like this content and want me to keep writing about it. Leave your impression and/or your comment.</p><p>You can follow me on Twitter. I'm always writing about Computer Science stuff.</p>]]><![CDATA[How Naive Bayes Classifiers Work – with Python Code Examples]]>https://blog.jayjaydev.com/how-naive-bayes-classifiers-work-with-python-code-exampleshttps://blog.jayjaydev.com/how-naive-bayes-classifiers-work-with-python-code-examplesFri, 06 Nov 2020 16:07:54 GMT<![CDATA[<p>Naive Bayes Classifiers (NBC) are simple yet powerful Machine Learning algorithms. They are based on conditional probability and Bayes's Theorem.</p><p>In this post, I explain "the trick" behind NBC and I'll give you an example that we can use to solve a classification problem.</p><p>In the next sections, I'll be talking about the math behind NBC. Feel free to skip those sections and go to the implementation part if you are not interested in the math.</p><p>In the implementation section, I'll show you a simple NBC algorithm. Then we'll use it to solve a classification problem. The task will be to determine whether a certain passenger on the Titanic survived the accident or not.</p><h2 id="conditional-probability">Conditional probability</h2><p>Before talking about the algorithm itself, let's talk about the simple math behind it. We need to understand what conditional probability is and how can we use Bayes's Theorem to calculate it.</p><p>Think about a fair die with six sides. What's the probability of getting a six when rolling the die? That's easy, it's 1/6. We have six possible and equally likely outcomes but we are interested in just one of them. So, 1/6 it is.</p><p>But what happens if I tell you that I have rolled the die already and the outcome is an even number? What's the probability that we have got a six now?</p><p>This time, the possible outcomes are just three because there are only three even numbers on the die. We are still interested in just one of those outcomes, so now the probability is greater: 1/3. What's the difference between both cases?</p><p>In the first case, we had no prior information about the outcome. Thus, we needed to consider every single possible result.</p><p>In the second case, we were told that the outcome was an even number, so we could reduce the space of possible outcomes to just the three even numbers that appear in a regular six-sided die.</p><p>In general, when calculating the probability of an event A, given the occurrence of another event B, we say we are calculating the conditional probability of A given B, or just the probability of A given B. We denote it by <code>P(A|B)</code>.</p><p>For example, the probability of getting a six given that the number we have got is even: <code>P(Six|Even) = 1/3</code>. Here we, denoted with Six the event of getting a six and with Even the event of getting an even number.</p><p>But, how do we calculate conditional probabilities? Is there a formula?</p><h2 id="how-to-calculate-conditional-probs-and-bayess-theorem">How to calculate conditional probs and Bayes's Theorem</h2><p>Now, I'll give you a couple of formulas to calculate conditional probs. I promise they won't be hard, and they are important if you want to understand the insights of the Machine Learning algorithms we'll be talking about later.</p><p>The probability of an event A given the occurrence of another event B can be calculated as follows:</p><pre><code>P(A|B) = P(A,B)/P(B)</code></pre><p>Where <code>P(A, B)</code> denotes the probability of both A and B occurring at the same time, and P(B) denotes the probability of B.</p><p>Notice that we need P(B) > 0 because it makes no sense to talk about the probability of A given B if the occurrence of B is not possible.</p><p>We can also calculate the probability of an event A, given the occurrence of multiple events B1, B2,..., Bn:</p><pre><code><span class="hljs-attribute">P</span>(A|B<span class="hljs-number">1</span>,B<span class="hljs-number">2</span>,...,Bn) = P(A,B<span class="hljs-number">1</span>,B<span class="hljs-number">2</span>,...,Bn)/P(B<span class="hljs-number">1</span>,B<span class="hljs-number">2</span>,...,Bn)</code></pre><p>There's another way of calculating conditional probs. This way is the so-called Bayes's Theorem.</p><pre><code><span class="hljs-attribute">P</span>(A|B) = P(B|A)P(A)/P(B)<span class="hljs-attribute">P</span>(A|B<span class="hljs-number">1</span>,B<span class="hljs-number">2</span>,...,Bn) = P(B<span class="hljs-number">1</span>,B<span class="hljs-number">2</span>,...,Bn|A)P(A)/P(B<span class="hljs-number">1</span>,B<span class="hljs-number">2</span>,...,Bn)</code></pre><p>Notice that we are calculating the probability of event A given event B, by inverting the order of occurrence of the events.</p><p>Now we suppose event A has occurred and we want to calculate the prob of event B (or events B1, B2,..., Bn in the second and more general example).</p><p>An important fact that can be derived from this Theorem is the formula to calculate <code>P(B1, B2,..., Bn, A)</code>. That's called the chain rule for probabilities.</p><pre><code>P(B1,B2,<span class="hljs-keyword">...</span>,Bn,A) = P(B1 | B2, B3, <span class="hljs-keyword">...</span>, Bn, A)P(B2,B3,<span class="hljs-keyword">...</span>,Bn,A)= P(B1 | B2, B3, <span class="hljs-keyword">...</span>, Bn, A)P(B2 | B3, B4, <span class="hljs-keyword">...</span>, Bn, A)P(B3, B4, <span class="hljs-keyword">...</span>, Bn, A)= P(B1 | B2, B3, <span class="hljs-keyword">...</span>, Bn, A)P(B2 | B3, B4, <span class="hljs-keyword">...</span>, Bn, A)...P(Bn | A)P(A)</code></pre><p>That's an ugly formula, isn't it? But under some conditions, we can make a workaround and avoid it.</p><p>Let's talk about the last concept we need to know to understand the algorithms.</p><h2 id="independence">Independence</h2><p>The last concept we are going to talk about is independence. We say events A and B are independent if</p><pre><code>P(A|B) = P(A)</code></pre><p>That means that the prob of event A is not affected by the occurrence of event B. A direct consequence is that <code>P(A,B) = P(A)P(B)</code>.</p><p>In plain English, this means that the prob of the occurrence of both A and B at the same time is equal to the product of the probs of events A and B occurring separately.</p><p>If A and B are independent, it also holds that:</p><pre><code>P(A,B<span class="hljs-params">|C) = P(A|</span>C)P(B<span class="hljs-params">|C)</span></code></pre><p>Now we are ready to talk about Naive Bayes Classifiers!</p><h2 id="naive-bayes-classifiers">Naive Bayes Classifiers</h2><p>Suppose we have a vector X of n features and we want to determine the class of that vector from a set of k classes y1, y2,..., yk. For example, if we want to determine whether it'll rain today or not.</p><p>We have two possible classes (k = 2): rain, not rain, and the length of the vector of features might be 3 (n = 3).</p><p>The first feature might be whether it is cloudy or sunny, the second feature could be whether humidity is high or low, and the third feature would be whether the temperature is high, medium, or low.</p><p>So, these could be possible feature vectors.</p><pre><code><span class="hljs-tag"><<span class="hljs-name">Cloudy,</span> <span class="hljs-attr">H_High</span>, <span class="hljs-attr">T_Low</span>></span><span class="hljs-tag"><<span class="hljs-name">Sunny,</span> <span class="hljs-attr">H_Low</span>, <span class="hljs-attr">T_Medium</span>></span><span class="hljs-tag"><<span class="hljs-name">Cloudy,</span> <span class="hljs-attr">H_Low</span>, <span class="hljs-attr">T_High</span>></span></code></pre><p>Our task is to determine whether it'll rain or not, given the weather features.</p><p>After learning about conditional probabilities, it seems natural to approach the problem by trying to calculate the prob of raining given the features:</p><pre><code><span class="hljs-attr">R</span> = P(Rain | Cloudy, H_High, T_Low)<span class="hljs-attr">NR</span> = P(NotRain | Cloudy, H_High, T_Low)</code></pre><p>If R > NR we answer that it'll rain, otherwise we say it won't.</p><p>In general, if we have k classes y1, y2, ..., yk, and a vector of n features <code>X = <X1, X2, ..., Xn></code>, we want to find the class yi that maximizes</p><pre><code><span class="hljs-attribute">P</span>(yi | X<span class="hljs-number">1</span>, X<span class="hljs-number">2</span>, ..., Xn) = P(X<span class="hljs-number">1</span>, X<span class="hljs-number">2</span>,..., Xn, yi)/P(X<span class="hljs-number">1</span>, X<span class="hljs-number">2</span>, ..., Xn)</code></pre><p>Notice that the denominator is constant and it does not depend on the class yi. So, we can ignore it and just focus on the numerator.</p><p>In a previous section, we saw how to calculate <code>P(X1, X2,..., Xn, yi)</code> by decomposing it in a product of conditional probabilities (the ugly formula):</p><pre><code><span class="hljs-attribute">P</span>(X<span class="hljs-number">1</span>, X<span class="hljs-number">2</span>,..., Xn, yi) = P(X<span class="hljs-number">1</span> | X<span class="hljs-number">2</span>,..., Xn, yi)P(X<span class="hljs-number">2</span> | X<span class="hljs-number">3</span>,..., Xn, yi)...P(Xn | yi)P(yi)</code></pre><p>Assuming all the features Xi are independent and using Bayes's Theorem, we can calculate the conditional probability as follows:</p><pre><code>P(yi | X1, X2,<span class="hljs-keyword">...</span>, Xn) = P(X1, X2,<span class="hljs-keyword">...</span>, Xn | yi)P(yi)/P(X1, X2, <span class="hljs-keyword">...</span>, Xn)= P(X1 | yi)P(X2 | yi)...P(Xn | yi)P(yi)/P(X1, X2, <span class="hljs-keyword">...</span>, Xn)</code></pre><p>And we just need to focus on the numerator.</p><p>By finding the class yi that maximizes the previous expression, we are classifying the input vector. But, how can we get all those probabilities?</p><h2 id="how-to-calculate-the-probabilities">How to calculate the probabilities</h2><p>When solving these kind of problems we need to have a set of previously classified examples.</p><p>For instance, in the problem of guessing whether it'll rain or not, we need to have several examples of feature vectors and their classifications that they would be obtained from past weather forecasts.</p><p>So, we would have something like this:</p><pre><code><span class="hljs-meta">...</span><Cloudy, H_High, T_Low> -> Rain<Sunny, H_Low, T_Medium> -> Not Rain<Cloudy, H_Low, T_High> -> Not Rain<span class="hljs-meta">...</span></code></pre><p>Suppose we need to classify a new vector . We need to calculate:</p><pre><code>P(Rain <span class="hljs-params">| Cloudy, H_Low, T_Low) = P(Cloudy |</span> H_Low, T_Low, Rain)P(H_Low <span class="hljs-params">| T_Low, Rain)P(T_Low |</span> Rain)P(Rain)/P(Cloudy, H_Low, T_Low)</code></pre><p>We get the previous expression by applying the definition of conditional probability and the chain rule. Remember we only need to focus on the numerator so we can drop the denominator.</p><p>We also need to calculate the prob for NotRain, but we can do this in a similar way.</p><p>We can find <code>P(Rain) = # Rain/Total</code>. That means counting the entries in the dataset that are classified with Rain and dividing that number by the size of the dataset.</p><p>To calculate <code>P(Cloudy | H_Low, T_Low, Rain)</code> we need to count all the entries that have the features H_Low, T_Low, and Cloudy. Those entries also need to be classified as Rain. Then, that number is divided by the total amount of data. We calculate the rest of the factors of the formula in a similar fashion.</p><p>Making those computations for every possible class is very expensive and slow. So we need to make assumptions about the problem that simplify the calculations.</p><p>Naive Bayes Classifiers assume that all the features are independent from each other. So we can rewrite our formula applying Bayes's Theorem and assuming independence between every pair of features:</p><pre><code>P(Rain <span class="hljs-params">| Cloudy, H_Low, T_Low) = P(Cloudy |</span> Rain)P(H_Low <span class="hljs-params">| Rain)P(T_Low |</span> Rain)P(Rain)/P(Cloudy, H_Low, T_Low)</code></pre><p>Now we calculate <code>P(Cloudy | Rain)</code> counting the number of entries that are classified as Rain and were Cloudy.</p><blockquote><p>The algorithm is called Naive because of this independence assumption. There are dependencies between the features most of the time. We can't say that in real life there isn't a dependency between the humidity and the temperature, for example. Naive Bayes Classifiers are also called Independence Bayes, or Simple Bayes.</p></blockquote><p>The general formula would be:</p><pre><code><span class="hljs-attribute">P</span>(yi | X<span class="hljs-number">1</span>, X<span class="hljs-number">2</span>, ..., Xn) = P(X<span class="hljs-number">1</span> | yi)P(X<span class="hljs-number">2</span> | yi)...P(Xn | yi)P(yi)/P(X<span class="hljs-number">1</span>, X<span class="hljs-number">2</span>, ..., Xn)</code></pre><p>Remember you can get rid of the denominator. We only calculate the numerator and answer the class that maximizes it.</p><p>Now, let's implement our NBC, and let's use it in a problem.</p><h2 id="lets-code">Let's code!</h2><p>I will show you an implementation of a simple NBC and then we'll see it in practice.</p><p>The problem we are going to solve is determining whether a passenger on the Titanic survived or not, given some features like their gender and their age.</p><p>Here you can see the implementation of a very simple NBC:</p><pre><code class="lang-python"><span class="hljs-class"><span class="hljs-keyword">class</span> <span class="hljs-title">NaiveBayesClassifier</span>:</span> <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">__init__</span>(<span class="hljs-params">self, X, y</span>):</span> <span class="hljs-string">''' X and y denotes the features and the target labels respectively '''</span> self.X, self.y = X, y self.N = len(self.X) <span class="hljs-comment"># Length of the training set</span> self.dim = len(self.X[<span class="hljs-number">0</span>]) <span class="hljs-comment"># Dimension of the vector of features</span> self.attrs = [[] <span class="hljs-keyword">for</span> _ <span class="hljs-keyword">in</span> range(self.dim)] <span class="hljs-comment"># Here we'll store the columns of the training set</span> self.output_dom = {} <span class="hljs-comment"># Output classes with the number of ocurrences in the training set. In this case we have only 2 classes</span> self.data = [] <span class="hljs-comment"># To store every row [Xi, yi]</span> <span class="hljs-keyword">for</span> i <span class="hljs-keyword">in</span> range(len(self.X)): <span class="hljs-keyword">for</span> j <span class="hljs-keyword">in</span> range(self.dim): <span class="hljs-comment"># if we have never seen this value for this attr before, </span> <span class="hljs-comment"># then we add it to the attrs array in the corresponding position</span> <span class="hljs-keyword">if</span> <span class="hljs-keyword">not</span> self.X[i][j] <span class="hljs-keyword">in</span> self.attrs[j]: self.attrs[j].append(self.X[i][j]) <span class="hljs-comment"># if we have never seen this output class before,</span> <span class="hljs-comment"># then we add it to the output_dom and count one occurrence for now</span> <span class="hljs-keyword">if</span> <span class="hljs-keyword">not</span> self.y[i] <span class="hljs-keyword">in</span> self.output_dom.keys(): self.output_dom[self.y[i]] = <span class="hljs-number">1</span> <span class="hljs-comment"># otherwise, we increment the occurrence of this output in the training set by 1</span> <span class="hljs-keyword">else</span>: self.output_dom[self.y[i]] += <span class="hljs-number">1</span> <span class="hljs-comment"># store the row</span> self.data.append([self.X[i], self.y[i]]) <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">classify</span>(<span class="hljs-params">self, entry</span>):</span> solve = <span class="hljs-literal">None</span> <span class="hljs-comment"># Final result</span> max_arg = <span class="hljs-number">-1</span> <span class="hljs-comment"># partial maximum</span> <span class="hljs-keyword">for</span> y <span class="hljs-keyword">in</span> self.output_dom.keys(): prob = self.output_dom[y]/self.N <span class="hljs-comment"># P(y)</span> <span class="hljs-keyword">for</span> i <span class="hljs-keyword">in</span> range(self.dim): cases = [x <span class="hljs-keyword">for</span> x <span class="hljs-keyword">in</span> self.data <span class="hljs-keyword">if</span> x[<span class="hljs-number">0</span>][i] == entry[i] <span class="hljs-keyword">and</span> x[<span class="hljs-number">1</span>] == y] <span class="hljs-comment"># all rows with Xi = xi</span> n = len(cases) prob *= n/self.N <span class="hljs-comment"># P *= P(Xi = xi)</span> <span class="hljs-comment"># if we have a greater prob for this output than the partial maximum...</span> <span class="hljs-keyword">if</span> prob > max_arg: max_arg = prob solve = y <span class="hljs-keyword">return</span> solve</code></pre><p>Here, we assume every feature has a discrete domain. That means they take a value from a finite set of possible values.</p><p>The same happens with classes. Notice that we store some data in the <strong>init</strong> method so we don't need to repeat some operations. The classification of a new entry is carried on in the classify method.</p><blockquote><p>This is a simple example of an implementation. In real-world applications, you don't need (and is better if you don't make) your own implementation. For example, the sklearn library in Python contains several good implementations of NBC's.</p></blockquote><p>Notice how easy it is to implement it!</p><p>Now, let's apply our new classifier to solve a problem. We have a dataset with a description of 887 passengers on the Titanic. We also can see whether a given passenger survived the tragedy or not.</p><p>So our task is to determine if another passenger that is not included in the training set made it or not.</p><p>In this example, I'll be using the <strong>pandas</strong> library to read and process the data. I don't use any other tool.</p><p>The data is stored in a file called titanic.csv, so the first step is to read the data and get an overview of it.</p><pre><code class="lang-python"><span class="hljs-keyword">import</span> pandas <span class="hljs-keyword">as</span> pddata = pd.read_csv(<span class="hljs-string">'titanic.csv'</span>)print(data.head())</code></pre><p>The output is:</p><pre><code><span class="hljs-attribute">Survived</span> Pclass Name \<span class="hljs-attribute">0</span> <span class="hljs-number">0</span> <span class="hljs-number">3</span> Mr. Owen Harris Braund <span class="hljs-attribute">1</span> <span class="hljs-number">1</span> <span class="hljs-number">1</span> Mrs. John Bradley (Florence Briggs Thayer) Cum... <span class="hljs-attribute">2</span> <span class="hljs-number">1</span> <span class="hljs-number">3</span> Miss. Laina Heikkinen <span class="hljs-attribute">3</span> <span class="hljs-number">1</span> <span class="hljs-number">1</span> Mrs. Jacques Heath (Lily May Peel) Futrelle <span class="hljs-attribute">4</span> <span class="hljs-number">0</span> <span class="hljs-number">3</span> Mr. William Henry Allen <span class="hljs-attribute">Sex</span> Age Siblings/Spouses Aboard Parents/Children Aboard Fare <span class="hljs-attribute">0</span> male <span class="hljs-number">22</span>.<span class="hljs-number">0</span> <span class="hljs-number">1</span> <span class="hljs-number">0</span> <span class="hljs-number">7</span>.<span class="hljs-number">2500</span> <span class="hljs-attribute">1</span> female <span class="hljs-number">38</span>.<span class="hljs-number">0</span> <span class="hljs-number">1</span> <span class="hljs-number">0</span> <span class="hljs-number">71</span>.<span class="hljs-number">2833</span> <span class="hljs-attribute">2</span> female <span class="hljs-number">26</span>.<span class="hljs-number">0</span> <span class="hljs-number">0</span> <span class="hljs-number">0</span> <span class="hljs-number">7</span>.<span class="hljs-number">9250</span> <span class="hljs-attribute">3</span> female <span class="hljs-number">35</span>.<span class="hljs-number">0</span> <span class="hljs-number">1</span> <span class="hljs-number">0</span> <span class="hljs-number">53</span>.<span class="hljs-number">1000</span> <span class="hljs-attribute">4</span> male <span class="hljs-number">35</span>.<span class="hljs-number">0</span> <span class="hljs-number">0</span> <span class="hljs-number">0</span> <span class="hljs-number">8</span>.<span class="hljs-number">0500</span></code></pre><p>Notice we have the Name of each passenger. We won't use that feature for our classifier because it is not significant for our problem. We'll also get rid of the Fare feature because it is continuous and our features need to be discrete.</p><blockquote><p>There are Naive Bayes Classifiers that support continuous features. For example, the Gaussian Naive Bayes Classifier.</p></blockquote><pre><code class="lang-python">y = list(map(<span class="hljs-keyword">lambda</span> v: <span class="hljs-string">'yes'</span> <span class="hljs-keyword">if</span> v == <span class="hljs-number">1</span> <span class="hljs-keyword">else</span> <span class="hljs-string">'no'</span>, data[<span class="hljs-string">'Survived'</span>].values)) <span class="hljs-comment"># target values as string</span><span class="hljs-comment"># We won't use the 'Name' nor the 'Fare' field</span>X = data[[<span class="hljs-string">'Pclass'</span>, <span class="hljs-string">'Sex'</span>, <span class="hljs-string">'Age'</span>, <span class="hljs-string">'Siblings/Spouses Aboard'</span>, <span class="hljs-string">'Parents/Children Aboard'</span>]].values <span class="hljs-comment"># features values</span></code></pre><p>Then, we need to separate our data set into a training set and a validation set. The latter is used to validate how well our algorithm is doing.</p><pre><code class="lang-python">print(len(y)) <span class="hljs-comment"># >> 887</span><span class="hljs-comment"># We'll take 600 examples to train and the rest to the validation process</span>y_train = y[:<span class="hljs-number">600</span>]y_val = y[<span class="hljs-number">600</span>:]X_train = X[:<span class="hljs-number">600</span>]X_val = X[<span class="hljs-number">600</span>:]</code></pre><p>We create our NBC with the training set and then classify every entry in the validation set.</p><p>We measure the accuracy of our algorithm by dividing the number of entries it correctly classified by the total number of entries in the validation set.</p><pre><code class="lang-python"><span class="hljs-comment">## Creating the Naive Bayes Classifier instance with the training data</span>nbc = NaiveBayesClassifier(X_train, y_train)total_cases = len(y_val) <span class="hljs-comment"># size of validation set</span><span class="hljs-comment"># Well classified examples and bad classified examples</span>good = <span class="hljs-number">0</span>bad = <span class="hljs-number">0</span><span class="hljs-keyword">for</span> i <span class="hljs-keyword">in</span> range(total_cases): predict = nbc.classify(X_val[i])<span class="hljs-comment"># print(y_val[i] + ' --------------- ' + predict)</span> <span class="hljs-keyword">if</span> y_val[i] == predict: good += <span class="hljs-number">1</span> <span class="hljs-keyword">else</span>: bad += <span class="hljs-number">1</span>print(<span class="hljs-string">'TOTAL EXAMPLES:'</span>, total_cases)print(<span class="hljs-string">'RIGHT:'</span>, good)print(<span class="hljs-string">'WRONG:'</span>, bad)print(<span class="hljs-string">'ACCURACY:'</span>, good/total_cases)</code></pre><p>The output:</p><pre><code><span class="hljs-attr">TOTAL EXAMPLES:</span> <span class="hljs-number">287</span><span class="hljs-attr">RIGHT:</span> <span class="hljs-number">200</span><span class="hljs-attr">WRONG:</span> <span class="hljs-number">87</span><span class="hljs-attr">ACCURACY:</span> <span class="hljs-number">0.6968641114982579</span></code></pre><p>It's not great but it's something. We can get about a 10% accuracy improvement if we get rid of other features like Siblings/Spouses Aboard and Parents/Children Aboard.</p><p>You can see a notebook with the code and the dataset here</p><h2 id="conclusions">Conclusions</h2><p>Today, we have neural networks and other complex and expensive ML algorithms all over the place.</p><p>NBCs are very simple algorithms that let us achieve good results in some classification problems without needing a lot of resources. They also scale very well, which means we can add a lot more features and the algorithm will still be fast and reliable.</p><p>Even in a case where NBCs were not a good fit for the problem we were trying to solve, they might be very useful as a baseline.</p><p>We could first try to solve the problem using an NBC with a few lines of code and little effort. Then we could try to achieve better results with more complex and expensive algorithms.</p><p>This process can save us a lot of time and gives us immediate feedback about whether complex algorithms are really worth it for our task.</p><p>In this article, you read about conditional probabilities, independence, and Bayes's Theorem. Those are the Mathematical concepts behind Naive Bayes Classifiers.</p><p>After that, we saw a simple implementation of an NBC and solved the problem of determining whether a passenger on the Titanic survived the accident.</p><p>I hope you found this article useful. You can read about Computer Science related topics in my personal blog and by following me on <a target="_blank" href="https://twitter.com/josejorgexl">Twitter</a>.</p>]]><![CDATA[<p>Naive Bayes Classifiers (NBC) are simple yet powerful Machine Learning algorithms. They are based on conditional probability and Bayes's Theorem.</p><p>In this post, I explain "the trick" behind NBC and I'll give you an example that we can use to solve a classification problem.</p><p>In the next sections, I'll be talking about the math behind NBC. Feel free to skip those sections and go to the implementation part if you are not interested in the math.</p><p>In the implementation section, I'll show you a simple NBC algorithm. Then we'll use it to solve a classification problem. The task will be to determine whether a certain passenger on the Titanic survived the accident or not.</p><h2 id="conditional-probability">Conditional probability</h2><p>Before talking about the algorithm itself, let's talk about the simple math behind it. We need to understand what conditional probability is and how can we use Bayes's Theorem to calculate it.</p><p>Think about a fair die with six sides. What's the probability of getting a six when rolling the die? That's easy, it's 1/6. We have six possible and equally likely outcomes but we are interested in just one of them. So, 1/6 it is.</p><p>But what happens if I tell you that I have rolled the die already and the outcome is an even number? What's the probability that we have got a six now?</p><p>This time, the possible outcomes are just three because there are only three even numbers on the die. We are still interested in just one of those outcomes, so now the probability is greater: 1/3. What's the difference between both cases?</p><p>In the first case, we had no prior information about the outcome. Thus, we needed to consider every single possible result.</p><p>In the second case, we were told that the outcome was an even number, so we could reduce the space of possible outcomes to just the three even numbers that appear in a regular six-sided die.</p><p>In general, when calculating the probability of an event A, given the occurrence of another event B, we say we are calculating the conditional probability of A given B, or just the probability of A given B. We denote it by <code>P(A|B)</code>.</p><p>For example, the probability of getting a six given that the number we have got is even: <code>P(Six|Even) = 1/3</code>. Here we, denoted with Six the event of getting a six and with Even the event of getting an even number.</p><p>But, how do we calculate conditional probabilities? Is there a formula?</p><h2 id="how-to-calculate-conditional-probs-and-bayess-theorem">How to calculate conditional probs and Bayes's Theorem</h2><p>Now, I'll give you a couple of formulas to calculate conditional probs. I promise they won't be hard, and they are important if you want to understand the insights of the Machine Learning algorithms we'll be talking about later.</p><p>The probability of an event A given the occurrence of another event B can be calculated as follows:</p><pre><code>P(A|B) = P(A,B)/P(B)</code></pre><p>Where <code>P(A, B)</code> denotes the probability of both A and B occurring at the same time, and P(B) denotes the probability of B.</p><p>Notice that we need P(B) > 0 because it makes no sense to talk about the probability of A given B if the occurrence of B is not possible.</p><p>We can also calculate the probability of an event A, given the occurrence of multiple events B1, B2,..., Bn:</p><pre><code><span class="hljs-attribute">P</span>(A|B<span class="hljs-number">1</span>,B<span class="hljs-number">2</span>,...,Bn) = P(A,B<span class="hljs-number">1</span>,B<span class="hljs-number">2</span>,...,Bn)/P(B<span class="hljs-number">1</span>,B<span class="hljs-number">2</span>,...,Bn)</code></pre><p>There's another way of calculating conditional probs. This way is the so-called Bayes's Theorem.</p><pre><code><span class="hljs-attribute">P</span>(A|B) = P(B|A)P(A)/P(B)<span class="hljs-attribute">P</span>(A|B<span class="hljs-number">1</span>,B<span class="hljs-number">2</span>,...,Bn) = P(B<span class="hljs-number">1</span>,B<span class="hljs-number">2</span>,...,Bn|A)P(A)/P(B<span class="hljs-number">1</span>,B<span class="hljs-number">2</span>,...,Bn)</code></pre><p>Notice that we are calculating the probability of event A given event B, by inverting the order of occurrence of the events.</p><p>Now we suppose event A has occurred and we want to calculate the prob of event B (or events B1, B2,..., Bn in the second and more general example).</p><p>An important fact that can be derived from this Theorem is the formula to calculate <code>P(B1, B2,..., Bn, A)</code>. That's called the chain rule for probabilities.</p><pre><code>P(B1,B2,<span class="hljs-keyword">...</span>,Bn,A) = P(B1 | B2, B3, <span class="hljs-keyword">...</span>, Bn, A)P(B2,B3,<span class="hljs-keyword">...</span>,Bn,A)= P(B1 | B2, B3, <span class="hljs-keyword">...</span>, Bn, A)P(B2 | B3, B4, <span class="hljs-keyword">...</span>, Bn, A)P(B3, B4, <span class="hljs-keyword">...</span>, Bn, A)= P(B1 | B2, B3, <span class="hljs-keyword">...</span>, Bn, A)P(B2 | B3, B4, <span class="hljs-keyword">...</span>, Bn, A)...P(Bn | A)P(A)</code></pre><p>That's an ugly formula, isn't it? But under some conditions, we can make a workaround and avoid it.</p><p>Let's talk about the last concept we need to know to understand the algorithms.</p><h2 id="independence">Independence</h2><p>The last concept we are going to talk about is independence. We say events A and B are independent if</p><pre><code>P(A|B) = P(A)</code></pre><p>That means that the prob of event A is not affected by the occurrence of event B. A direct consequence is that <code>P(A,B) = P(A)P(B)</code>.</p><p>In plain English, this means that the prob of the occurrence of both A and B at the same time is equal to the product of the probs of events A and B occurring separately.</p><p>If A and B are independent, it also holds that:</p><pre><code>P(A,B<span class="hljs-params">|C) = P(A|</span>C)P(B<span class="hljs-params">|C)</span></code></pre><p>Now we are ready to talk about Naive Bayes Classifiers!</p><h2 id="naive-bayes-classifiers">Naive Bayes Classifiers</h2><p>Suppose we have a vector X of n features and we want to determine the class of that vector from a set of k classes y1, y2,..., yk. For example, if we want to determine whether it'll rain today or not.</p><p>We have two possible classes (k = 2): rain, not rain, and the length of the vector of features might be 3 (n = 3).</p><p>The first feature might be whether it is cloudy or sunny, the second feature could be whether humidity is high or low, and the third feature would be whether the temperature is high, medium, or low.</p><p>So, these could be possible feature vectors.</p><pre><code><span class="hljs-tag"><<span class="hljs-name">Cloudy,</span> <span class="hljs-attr">H_High</span>, <span class="hljs-attr">T_Low</span>></span><span class="hljs-tag"><<span class="hljs-name">Sunny,</span> <span class="hljs-attr">H_Low</span>, <span class="hljs-attr">T_Medium</span>></span><span class="hljs-tag"><<span class="hljs-name">Cloudy,</span> <span class="hljs-attr">H_Low</span>, <span class="hljs-attr">T_High</span>></span></code></pre><p>Our task is to determine whether it'll rain or not, given the weather features.</p><p>After learning about conditional probabilities, it seems natural to approach the problem by trying to calculate the prob of raining given the features:</p><pre><code><span class="hljs-attr">R</span> = P(Rain | Cloudy, H_High, T_Low)<span class="hljs-attr">NR</span> = P(NotRain | Cloudy, H_High, T_Low)</code></pre><p>If R > NR we answer that it'll rain, otherwise we say it won't.</p><p>In general, if we have k classes y1, y2, ..., yk, and a vector of n features <code>X = <X1, X2, ..., Xn></code>, we want to find the class yi that maximizes</p><pre><code><span class="hljs-attribute">P</span>(yi | X<span class="hljs-number">1</span>, X<span class="hljs-number">2</span>, ..., Xn) = P(X<span class="hljs-number">1</span>, X<span class="hljs-number">2</span>,..., Xn, yi)/P(X<span class="hljs-number">1</span>, X<span class="hljs-number">2</span>, ..., Xn)</code></pre><p>Notice that the denominator is constant and it does not depend on the class yi. So, we can ignore it and just focus on the numerator.</p><p>In a previous section, we saw how to calculate <code>P(X1, X2,..., Xn, yi)</code> by decomposing it in a product of conditional probabilities (the ugly formula):</p><pre><code><span class="hljs-attribute">P</span>(X<span class="hljs-number">1</span>, X<span class="hljs-number">2</span>,..., Xn, yi) = P(X<span class="hljs-number">1</span> | X<span class="hljs-number">2</span>,..., Xn, yi)P(X<span class="hljs-number">2</span> | X<span class="hljs-number">3</span>,..., Xn, yi)...P(Xn | yi)P(yi)</code></pre><p>Assuming all the features Xi are independent and using Bayes's Theorem, we can calculate the conditional probability as follows:</p><pre><code>P(yi | X1, X2,<span class="hljs-keyword">...</span>, Xn) = P(X1, X2,<span class="hljs-keyword">...</span>, Xn | yi)P(yi)/P(X1, X2, <span class="hljs-keyword">...</span>, Xn)= P(X1 | yi)P(X2 | yi)...P(Xn | yi)P(yi)/P(X1, X2, <span class="hljs-keyword">...</span>, Xn)</code></pre><p>And we just need to focus on the numerator.</p><p>By finding the class yi that maximizes the previous expression, we are classifying the input vector. But, how can we get all those probabilities?</p><h2 id="how-to-calculate-the-probabilities">How to calculate the probabilities</h2><p>When solving these kind of problems we need to have a set of previously classified examples.</p><p>For instance, in the problem of guessing whether it'll rain or not, we need to have several examples of feature vectors and their classifications that they would be obtained from past weather forecasts.</p><p>So, we would have something like this:</p><pre><code><span class="hljs-meta">...</span><Cloudy, H_High, T_Low> -> Rain<Sunny, H_Low, T_Medium> -> Not Rain<Cloudy, H_Low, T_High> -> Not Rain<span class="hljs-meta">...</span></code></pre><p>Suppose we need to classify a new vector . We need to calculate:</p><pre><code>P(Rain <span class="hljs-params">| Cloudy, H_Low, T_Low) = P(Cloudy |</span> H_Low, T_Low, Rain)P(H_Low <span class="hljs-params">| T_Low, Rain)P(T_Low |</span> Rain)P(Rain)/P(Cloudy, H_Low, T_Low)</code></pre><p>We get the previous expression by applying the definition of conditional probability and the chain rule. Remember we only need to focus on the numerator so we can drop the denominator.</p><p>We also need to calculate the prob for NotRain, but we can do this in a similar way.</p><p>We can find <code>P(Rain) = # Rain/Total</code>. That means counting the entries in the dataset that are classified with Rain and dividing that number by the size of the dataset.</p><p>To calculate <code>P(Cloudy | H_Low, T_Low, Rain)</code> we need to count all the entries that have the features H_Low, T_Low, and Cloudy. Those entries also need to be classified as Rain. Then, that number is divided by the total amount of data. We calculate the rest of the factors of the formula in a similar fashion.</p><p>Making those computations for every possible class is very expensive and slow. So we need to make assumptions about the problem that simplify the calculations.</p><p>Naive Bayes Classifiers assume that all the features are independent from each other. So we can rewrite our formula applying Bayes's Theorem and assuming independence between every pair of features:</p><pre><code>P(Rain <span class="hljs-params">| Cloudy, H_Low, T_Low) = P(Cloudy |</span> Rain)P(H_Low <span class="hljs-params">| Rain)P(T_Low |</span> Rain)P(Rain)/P(Cloudy, H_Low, T_Low)</code></pre><p>Now we calculate <code>P(Cloudy | Rain)</code> counting the number of entries that are classified as Rain and were Cloudy.</p><blockquote><p>The algorithm is called Naive because of this independence assumption. There are dependencies between the features most of the time. We can't say that in real life there isn't a dependency between the humidity and the temperature, for example. Naive Bayes Classifiers are also called Independence Bayes, or Simple Bayes.</p></blockquote><p>The general formula would be:</p><pre><code><span class="hljs-attribute">P</span>(yi | X<span class="hljs-number">1</span>, X<span class="hljs-number">2</span>, ..., Xn) = P(X<span class="hljs-number">1</span> | yi)P(X<span class="hljs-number">2</span> | yi)...P(Xn | yi)P(yi)/P(X<span class="hljs-number">1</span>, X<span class="hljs-number">2</span>, ..., Xn)</code></pre><p>Remember you can get rid of the denominator. We only calculate the numerator and answer the class that maximizes it.</p><p>Now, let's implement our NBC, and let's use it in a problem.</p><h2 id="lets-code">Let's code!</h2><p>I will show you an implementation of a simple NBC and then we'll see it in practice.</p><p>The problem we are going to solve is determining whether a passenger on the Titanic survived or not, given some features like their gender and their age.</p><p>Here you can see the implementation of a very simple NBC:</p><pre><code class="lang-python"><span class="hljs-class"><span class="hljs-keyword">class</span> <span class="hljs-title">NaiveBayesClassifier</span>:</span> <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">__init__</span>(<span class="hljs-params">self, X, y</span>):</span> <span class="hljs-string">''' X and y denotes the features and the target labels respectively '''</span> self.X, self.y = X, y self.N = len(self.X) <span class="hljs-comment"># Length of the training set</span> self.dim = len(self.X[<span class="hljs-number">0</span>]) <span class="hljs-comment"># Dimension of the vector of features</span> self.attrs = [[] <span class="hljs-keyword">for</span> _ <span class="hljs-keyword">in</span> range(self.dim)] <span class="hljs-comment"># Here we'll store the columns of the training set</span> self.output_dom = {} <span class="hljs-comment"># Output classes with the number of ocurrences in the training set. In this case we have only 2 classes</span> self.data = [] <span class="hljs-comment"># To store every row [Xi, yi]</span> <span class="hljs-keyword">for</span> i <span class="hljs-keyword">in</span> range(len(self.X)): <span class="hljs-keyword">for</span> j <span class="hljs-keyword">in</span> range(self.dim): <span class="hljs-comment"># if we have never seen this value for this attr before, </span> <span class="hljs-comment"># then we add it to the attrs array in the corresponding position</span> <span class="hljs-keyword">if</span> <span class="hljs-keyword">not</span> self.X[i][j] <span class="hljs-keyword">in</span> self.attrs[j]: self.attrs[j].append(self.X[i][j]) <span class="hljs-comment"># if we have never seen this output class before,</span> <span class="hljs-comment"># then we add it to the output_dom and count one occurrence for now</span> <span class="hljs-keyword">if</span> <span class="hljs-keyword">not</span> self.y[i] <span class="hljs-keyword">in</span> self.output_dom.keys(): self.output_dom[self.y[i]] = <span class="hljs-number">1</span> <span class="hljs-comment"># otherwise, we increment the occurrence of this output in the training set by 1</span> <span class="hljs-keyword">else</span>: self.output_dom[self.y[i]] += <span class="hljs-number">1</span> <span class="hljs-comment"># store the row</span> self.data.append([self.X[i], self.y[i]]) <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">classify</span>(<span class="hljs-params">self, entry</span>):</span> solve = <span class="hljs-literal">None</span> <span class="hljs-comment"># Final result</span> max_arg = <span class="hljs-number">-1</span> <span class="hljs-comment"># partial maximum</span> <span class="hljs-keyword">for</span> y <span class="hljs-keyword">in</span> self.output_dom.keys(): prob = self.output_dom[y]/self.N <span class="hljs-comment"># P(y)</span> <span class="hljs-keyword">for</span> i <span class="hljs-keyword">in</span> range(self.dim): cases = [x <span class="hljs-keyword">for</span> x <span class="hljs-keyword">in</span> self.data <span class="hljs-keyword">if</span> x[<span class="hljs-number">0</span>][i] == entry[i] <span class="hljs-keyword">and</span> x[<span class="hljs-number">1</span>] == y] <span class="hljs-comment"># all rows with Xi = xi</span> n = len(cases) prob *= n/self.N <span class="hljs-comment"># P *= P(Xi = xi)</span> <span class="hljs-comment"># if we have a greater prob for this output than the partial maximum...</span> <span class="hljs-keyword">if</span> prob > max_arg: max_arg = prob solve = y <span class="hljs-keyword">return</span> solve</code></pre><p>Here, we assume every feature has a discrete domain. That means they take a value from a finite set of possible values.</p><p>The same happens with classes. Notice that we store some data in the <strong>init</strong> method so we don't need to repeat some operations. The classification of a new entry is carried on in the classify method.</p><blockquote><p>This is a simple example of an implementation. In real-world applications, you don't need (and is better if you don't make) your own implementation. For example, the sklearn library in Python contains several good implementations of NBC's.</p></blockquote><p>Notice how easy it is to implement it!</p><p>Now, let's apply our new classifier to solve a problem. We have a dataset with a description of 887 passengers on the Titanic. We also can see whether a given passenger survived the tragedy or not.</p><p>So our task is to determine if another passenger that is not included in the training set made it or not.</p><p>In this example, I'll be using the <strong>pandas</strong> library to read and process the data. I don't use any other tool.</p><p>The data is stored in a file called titanic.csv, so the first step is to read the data and get an overview of it.</p><pre><code class="lang-python"><span class="hljs-keyword">import</span> pandas <span class="hljs-keyword">as</span> pddata = pd.read_csv(<span class="hljs-string">'titanic.csv'</span>)print(data.head())</code></pre><p>The output is:</p><pre><code><span class="hljs-attribute">Survived</span> Pclass Name \<span class="hljs-attribute">0</span> <span class="hljs-number">0</span> <span class="hljs-number">3</span> Mr. Owen Harris Braund <span class="hljs-attribute">1</span> <span class="hljs-number">1</span> <span class="hljs-number">1</span> Mrs. John Bradley (Florence Briggs Thayer) Cum... <span class="hljs-attribute">2</span> <span class="hljs-number">1</span> <span class="hljs-number">3</span> Miss. Laina Heikkinen <span class="hljs-attribute">3</span> <span class="hljs-number">1</span> <span class="hljs-number">1</span> Mrs. Jacques Heath (Lily May Peel) Futrelle <span class="hljs-attribute">4</span> <span class="hljs-number">0</span> <span class="hljs-number">3</span> Mr. William Henry Allen <span class="hljs-attribute">Sex</span> Age Siblings/Spouses Aboard Parents/Children Aboard Fare <span class="hljs-attribute">0</span> male <span class="hljs-number">22</span>.<span class="hljs-number">0</span> <span class="hljs-number">1</span> <span class="hljs-number">0</span> <span class="hljs-number">7</span>.<span class="hljs-number">2500</span> <span class="hljs-attribute">1</span> female <span class="hljs-number">38</span>.<span class="hljs-number">0</span> <span class="hljs-number">1</span> <span class="hljs-number">0</span> <span class="hljs-number">71</span>.<span class="hljs-number">2833</span> <span class="hljs-attribute">2</span> female <span class="hljs-number">26</span>.<span class="hljs-number">0</span> <span class="hljs-number">0</span> <span class="hljs-number">0</span> <span class="hljs-number">7</span>.<span class="hljs-number">9250</span> <span class="hljs-attribute">3</span> female <span class="hljs-number">35</span>.<span class="hljs-number">0</span> <span class="hljs-number">1</span> <span class="hljs-number">0</span> <span class="hljs-number">53</span>.<span class="hljs-number">1000</span> <span class="hljs-attribute">4</span> male <span class="hljs-number">35</span>.<span class="hljs-number">0</span> <span class="hljs-number">0</span> <span class="hljs-number">0</span> <span class="hljs-number">8</span>.<span class="hljs-number">0500</span></code></pre><p>Notice we have the Name of each passenger. We won't use that feature for our classifier because it is not significant for our problem. We'll also get rid of the Fare feature because it is continuous and our features need to be discrete.</p><blockquote><p>There are Naive Bayes Classifiers that support continuous features. For example, the Gaussian Naive Bayes Classifier.</p></blockquote><pre><code class="lang-python">y = list(map(<span class="hljs-keyword">lambda</span> v: <span class="hljs-string">'yes'</span> <span class="hljs-keyword">if</span> v == <span class="hljs-number">1</span> <span class="hljs-keyword">else</span> <span class="hljs-string">'no'</span>, data[<span class="hljs-string">'Survived'</span>].values)) <span class="hljs-comment"># target values as string</span><span class="hljs-comment"># We won't use the 'Name' nor the 'Fare' field</span>X = data[[<span class="hljs-string">'Pclass'</span>, <span class="hljs-string">'Sex'</span>, <span class="hljs-string">'Age'</span>, <span class="hljs-string">'Siblings/Spouses Aboard'</span>, <span class="hljs-string">'Parents/Children Aboard'</span>]].values <span class="hljs-comment"># features values</span></code></pre><p>Then, we need to separate our data set into a training set and a validation set. The latter is used to validate how well our algorithm is doing.</p><pre><code class="lang-python">print(len(y)) <span class="hljs-comment"># >> 887</span><span class="hljs-comment"># We'll take 600 examples to train and the rest to the validation process</span>y_train = y[:<span class="hljs-number">600</span>]y_val = y[<span class="hljs-number">600</span>:]X_train = X[:<span class="hljs-number">600</span>]X_val = X[<span class="hljs-number">600</span>:]</code></pre><p>We create our NBC with the training set and then classify every entry in the validation set.</p><p>We measure the accuracy of our algorithm by dividing the number of entries it correctly classified by the total number of entries in the validation set.</p><pre><code class="lang-python"><span class="hljs-comment">## Creating the Naive Bayes Classifier instance with the training data</span>nbc = NaiveBayesClassifier(X_train, y_train)total_cases = len(y_val) <span class="hljs-comment"># size of validation set</span><span class="hljs-comment"># Well classified examples and bad classified examples</span>good = <span class="hljs-number">0</span>bad = <span class="hljs-number">0</span><span class="hljs-keyword">for</span> i <span class="hljs-keyword">in</span> range(total_cases): predict = nbc.classify(X_val[i])<span class="hljs-comment"># print(y_val[i] + ' --------------- ' + predict)</span> <span class="hljs-keyword">if</span> y_val[i] == predict: good += <span class="hljs-number">1</span> <span class="hljs-keyword">else</span>: bad += <span class="hljs-number">1</span>print(<span class="hljs-string">'TOTAL EXAMPLES:'</span>, total_cases)print(<span class="hljs-string">'RIGHT:'</span>, good)print(<span class="hljs-string">'WRONG:'</span>, bad)print(<span class="hljs-string">'ACCURACY:'</span>, good/total_cases)</code></pre><p>The output:</p><pre><code><span class="hljs-attr">TOTAL EXAMPLES:</span> <span class="hljs-number">287</span><span class="hljs-attr">RIGHT:</span> <span class="hljs-number">200</span><span class="hljs-attr">WRONG:</span> <span class="hljs-number">87</span><span class="hljs-attr">ACCURACY:</span> <span class="hljs-number">0.6968641114982579</span></code></pre><p>It's not great but it's something. We can get about a 10% accuracy improvement if we get rid of other features like Siblings/Spouses Aboard and Parents/Children Aboard.</p><p>You can see a notebook with the code and the dataset here</p><h2 id="conclusions">Conclusions</h2><p>Today, we have neural networks and other complex and expensive ML algorithms all over the place.</p><p>NBCs are very simple algorithms that let us achieve good results in some classification problems without needing a lot of resources. They also scale very well, which means we can add a lot more features and the algorithm will still be fast and reliable.</p><p>Even in a case where NBCs were not a good fit for the problem we were trying to solve, they might be very useful as a baseline.</p><p>We could first try to solve the problem using an NBC with a few lines of code and little effort. Then we could try to achieve better results with more complex and expensive algorithms.</p><p>This process can save us a lot of time and gives us immediate feedback about whether complex algorithms are really worth it for our task.</p><p>In this article, you read about conditional probabilities, independence, and Bayes's Theorem. Those are the Mathematical concepts behind Naive Bayes Classifiers.</p><p>After that, we saw a simple implementation of an NBC and solved the problem of determining whether a passenger on the Titanic survived the accident.</p><p>I hope you found this article useful. You can read about Computer Science related topics in my personal blog and by following me on <a target="_blank" href="https://twitter.com/josejorgexl">Twitter</a>.</p>]]><![CDATA[Improving linked lists]]>https://blog.jayjaydev.com/improving-linked-listshttps://blog.jayjaydev.com/improving-linked-listsSun, 01 Nov 2020 23:09:27 GMT<![CDATA[<p>You know the situation is tricky when you need more than just arrays and dictionaries. That was my case a couple of months ago. I needed an indexed data structure that supported insertion and removal operations as fast as possible.</p><p>With an array, I can get the i-th element really fast (this will be called the <em>index</em> operation. I provide an index <code>i</code> and get the <code>i-th</code> element of the list), no matter what the value of i is. But when it comes to inserting or removing an element from the middle of the array things change a little bit.</p><p>On the other hand, linked lists offer a slower index operation. We need to traverse a long segment of the list to get the element we are looking for. But we can make faster insertions and deletions.</p><p>In this article, I will write about a solution I came up with. This is a "new" data structure that somehow combines arrays and linked lists to get a middle point that suited my requirements. I don't know whether this is a truly novel data structure. Furthermore, I don't think it is an awesome idea. I just used some clever and relatively popular techniques to achieve my goal.</p><p>The next section is about the differences between arrays and linked lists, their respective strengths, and their weaknesses. If you are just interested in the data structure specifics, you can skip it. Then I'll talk about the core idea, and later, I'll give some implementation details.</p><blockquote><p>IMPORTANT! I assume you have some basic knowledge of algorithmic complexity and big-O notation.</p></blockquote><h2 id="heading-arrays-vs-linked-lists">Arrays vs. Linked Lists</h2><p>Arrays are contiguous memory slots to store sequences of data. We arrange some elements in such a way that we keep a reference to every single one of them. For example, if we have an array that contains numbers from 1 to 10, we have a reference to every memory slot that contains each of those ten numbers. That allows us to get any element by the index it has in the array really fast.</p><pre><code class="lang-python"><span class="hljs-comment"># Example of array </span>array = [<span class="hljs-number">1</span>, <span class="hljs-number">2</span>, <span class="hljs-number">3</span>, <span class="hljs-number">4</span>, <span class="hljs-number">5</span>, <span class="hljs-number">6</span>, <span class="hljs-number">7</span>, <span class="hljs-number">8</span>, <span class="hljs-number">9</span>, <span class="hljs-number">10</span>]array[<span class="hljs-number">3</span>]<span class="hljs-comment"># >> 4</span>array[<span class="hljs-number">0</span>]<span class="hljs-comment"># >> 1</span></code></pre><p>We have an implicit mapping from index to element. No matter the value of the index, the operation will always be equally fast. That means that doing <code>array[0]</code> is as fast as doing <code>array[9]</code> because we keep all the references at the same time. That's the most useful feature of arrays, we have access to any position in constant time, no matter the size of the array, nor the position itself.</p><p>This great feature is also the reason we can't insert or delete elements with the same ease. We need to keep elements in contiguous positions and maintain the references to them. When deleting an element, we are creating holes in the middle of the array, and that is not allowed. So we need to fix the references manually and fill the holes. The same happens with insertions, in this case, we need to create the hole first and then, fill it with the new element.</p><p>Thus, to insert/delete an element, we need to rearrange the array. If we have an array with <code>N</code> elements, we'll need to rearrange <code>O(N)</code> of those elements when doing an insertion/deletion. I needed to do it faster.</p><p>Linked lists don't maintain all those references in the same way. They are composed of nodes that store an element but also the reference to the next node. This way we only maintain a direct reference to the first node of the list (called the head), this node has a reference to the second one, the second one has a reference to the third one, and so on. In the case of doubly-linked lists, every node has a reference to both the next and the previous node, and we also keep a direct reference to the tail of the list (the last node).</p><p>Thus, looking for an element in a certain position implies to traverse <code>O(N)</code> nodes in the worst case. The good thing is that we can insert or delete an element without worrying about the holes because we only need to change a few references. But I needed to have access to positions in the lists in a faster way.</p><p>I needed to make faster insertion/deletion (better than <code>O(N)</code>), but yet be able to retrieve the <code>i-th</code> element of the list in less than <code>O(N)</code> operations as well. Is important to note that when I say insertion I mean an operation that receives an element and <strong>a position</strong>, and then inserts the element in that position. The delete operation is defined in the same way.</p><p>In the next section, I'll introduce a data structure that lets us fulfill such requirements.</p><h2 id="heading-an-indexed-linked-list">An Indexed Linked List</h2><p>As we saw in the previous section, the "problem" with arrays is that having all those references make the insertion/deletion slow. But the few references of linked lists are a problem as well, because we need to make a lot of operations to get the element at a given position. Thus, it seems reasonable to think about finding a middle point in the number of references.</p><p>First of all, I need to say that no data structure lets us make all the index, insert, and delete operations in constant time. So, a "perfect world" is out of the discussion. Let's move on.</p><p>I was thinking about having a doubly-linked list with more direct references. Instead of just keeping a reference to the first and the last node, I will keep also references to inner nodes, but how many references would I need? If I maintained just a few of them, the index operation might be slow, but if I maintained too many references, then insertions and deletions would take more time.</p><p>After thinking for a while, I came up with the right number of references: <code>sqrt(N)</code> (here <code>sqrt</code> stands for the square root operation). The idea is to keep a list of references, the length of that list will be <code>sqrt(N)</code>. The other important property is that every referenced node will be at a distance of <code>sqrt(N)</code> from the next node in the reference list.</p><p>For example, suppose we have a doubly-linked list with the numbers <code>1, 2, 3, 4, 5</code>. Then our reference list will contain two references because of <code>2 < sqrt(5) < 3</code>. Also, every node referenced in the list will be at a distance of <code>sqrt(5)</code> of the other node referenced in the list. So a possible list will be one that contains the second and the fourth node. Another possibility would be a list with the third and the fifth nodes. In both cases, the distance between nodes is less than <code>sqrt(5)</code>.</p><blockquote><p>When I say <code>sqrt(N)</code> I mean the largest integer <code>n</code> such that <code>n <= sqrt(N)</code></p></blockquote><p>By keeping such a reference list we can make all the index, insert, and delete operations in <code>O(sqrt(N))</code>. Note that we got a slower index operation to achieve a faster insertion/deletion.</p><blockquote><p>By the way, I borrowed some ideas from a technique called <strong>sqrt decomposition</strong>. You can check it out <a target="_blank" href="https://cp-algorithms.com/data_structures/sqrt_decomposition.html">here</a>.</p></blockquote><p>Now I'm going to explain the insert, delete, and index operations.</p><h2 id="heading-inserting-an-element">Inserting an element</h2><p>As I said, I maintain a list of selected nodes. The length of this list is always <code>sqrt(N)</code> and the distance between two consecutive selected nodes in the actual list will be also <code>sqrt(N)</code>.</p><p>Every time we insert a new node in the list we need to check whether the new length of the list has a new integer <code>sqrt(N)</code>. In that case, we need to update the selected nodes list. It would be like:</p><pre><code class="lang-python"><span class="hljs-keyword">import</span> math<span class="hljs-class"><span class="hljs-keyword">class</span> <span class="hljs-title">IndexedLinkedListNode</span>:</span> <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">__init__</span>(<span class="hljs-params">self, content, prev=None, next=None</span>):</span> self.content = content self.prev = prev self.next = next <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">__eq__</span>(<span class="hljs-params">self, other</span>):</span> <span class="hljs-keyword">return</span> self.content == other.content<span class="hljs-class"><span class="hljs-keyword">class</span> <span class="hljs-title">IndexedLinkedList</span>:</span> <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">__init__</span>(<span class="hljs-params">self, elements=[]</span>):</span> self.selected_nodes = [] self.length = <span class="hljs-number">0</span> self.head = <span class="hljs-literal">None</span> self.tail = <span class="hljs-literal">None</span> <span class="hljs-keyword">for</span> e <span class="hljs-keyword">in</span> elements: self.insert(e) <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">insert</span>(<span class="hljs-params">self, element</span>):</span> <span class="hljs-string">''' Inserts an element at the end of the list '''</span> new_node = IndexedLinkedListNode(element) sqlen = math.floor(math.sqrt(self.length)) <span class="hljs-keyword">if</span> self.length == <span class="hljs-number">0</span>: self.selected_nodes.append(new_node) self.head = new_node self.tail = new_node <span class="hljs-keyword">else</span>: self.tail.next = new_node new_node.prev = self.tail self.tail = new_node self.length += <span class="hljs-number">1</span> <span class="hljs-keyword">if</span> math.floor(math.sqrt(self.length)) > sqlen: self.update_list_cause_insertion(new_node) <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">update_list_cause_insertion</span>(<span class="hljs-params">self, new_node</span>):</span> <span class="hljs-string">''' Here we make every selected node become its next node in the list and add the new node to the list '''</span> self.selected_nodes = list(map(<span class="hljs-keyword">lambda</span> node : node.next, self.selected_nodes)) self.selected_nodes.append(new_node) <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">insert_at</span>(<span class="hljs-params">self, element, index</span>):</span> <span class="hljs-string">''' Here we insert the new node before the node that is in the specified position. You can implement yourself a very similar method to insert the new element AFTER the specified position. '''</span> new_node = IndexedLinkedListNode(element) target = self[index] sqlen = math.floor(math.sqrt(self.length)) self.length += <span class="hljs-number">1</span> <span class="hljs-keyword">if</span> index == <span class="hljs-number">0</span>: self.head.prev = new_node new_node.next = self.head self.head = new_node <span class="hljs-keyword">else</span>: target.prev.next = new_node new_node.prev = target.prev new_node.next = target target.prev = new_node <span class="hljs-keyword">if</span> math.floor(math.sqrt(self.length)) > sqlen: self.update_list_cause_insertion(self.tail)</code></pre><p>It can be proved that we always obtain a selected nodes list of length <code>sqrt(N)</code> and with at most <code>sqrt(N)</code> distance between any pair of consecutive nodes. I'm going to skip the demonstration to keep the article as simple as possible.</p><h2 id="heading-deletion">Deletion</h2><p>This is a very similar method. Is like doing the inverse operation. We just need to take care of some corner cases like deleting the only node in the list and deleting the first or the last node.</p><pre><code class="lang-python">... <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">remove_at</span>(<span class="hljs-params">self, index</span>):</span> target = self[index] sqlen = math.floor(math.sqrt(self.length)) selected_pos = <span class="hljs-literal">None</span> <span class="hljs-keyword">try</span>: selected_pos = self.selected_nodes.index(target) <span class="hljs-keyword">except</span>: <span class="hljs-keyword">pass</span> <span class="hljs-keyword">if</span> self.length <= <span class="hljs-number">1</span>: self.head = <span class="hljs-literal">None</span> self.tail = <span class="hljs-literal">None</span> self.selected_nodes = [] self.length = max(<span class="hljs-number">0</span>, self.length - <span class="hljs-number">1</span>) <span class="hljs-keyword">return</span> <span class="hljs-keyword">if</span> target.prev: target.prev.next = target.next <span class="hljs-keyword">else</span>: self.head = target.next <span class="hljs-keyword">if</span> target.next: target.next.prev = target.prev <span class="hljs-keyword">else</span>: self.tail = target.prev self.length -= <span class="hljs-number">1</span> <span class="hljs-keyword">if</span> selected_pos: <span class="hljs-keyword">for</span> selected <span class="hljs-keyword">in</span> self.selected_nodes[selected_pos:]: selected = selected.next <span class="hljs-keyword">if</span> math.floor(math.sqrt(self.length)) < sqlen: self.update_list_cause_removal() <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">update_list_cause_removal</span>(<span class="hljs-params">self</span>):</span> <span class="hljs-string">''' Here we make every selected node become its prev node in the list and remove the last selected node '''</span> self.selected_nodes = self.selected_nodes[:<span class="hljs-number">-1</span>] self.selected_nodes = list(map(<span class="hljs-keyword">lambda</span> node : node.prev, self.selected_nodes))...</code></pre><p>Until here we have achieved the requirements for insertion and deletion methods. But that is assuming that the index operation (<code>self[index]</code>) could be done also in the required time.</p><h2 id="heading-index">Index</h2><p>Let's see how to implement the index operation. This is the last piece of the puzzle. It is a trickier method that would require more explanation to understand the insights. As I said, I would like to keep this article as simple as possible so I'm not going to explain too many details about this operation.</p><pre><code class="lang-python">... <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">__getitem__</span>(<span class="hljs-params">self, index</span>):</span> <span class="hljs-keyword">if</span> len(self.selected_nodes) == <span class="hljs-number">0</span> <span class="hljs-keyword">or</span> index < <span class="hljs-number">0</span> <span class="hljs-keyword">or</span> index >= self.length: <span class="hljs-keyword">return</span> <span class="hljs-literal">None</span> first_index = math.floor(math.sqrt(self.length)) - <span class="hljs-number">1</span> <span class="hljs-comment"># index of the first selected node</span> first_node = self.selected_nodes[<span class="hljs-number">0</span>] <span class="hljs-keyword">if</span> index <= first_index: <span class="hljs-keyword">return</span> self.prevn(first_node, first_index - index) aux_idx = index - first_index target_k = self.integer_positive_root(aux_idx) target_index = target_k**<span class="hljs-number">2</span> + target_k + first_index index -= target_index <span class="hljs-keyword">return</span> self.nextn(self.selected_nodes[target_k], index) <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">integer_positive_root</span>(<span class="hljs-params">self, index</span>):</span> <span class="hljs-string">''' Solves the equation K^2 + K - index '''</span> <span class="hljs-keyword">return</span> math.floor((<span class="hljs-number">-1</span> + math.sqrt(<span class="hljs-number">1</span> + <span class="hljs-number">4</span>*index))/<span class="hljs-number">2</span>) <span class="hljs-comment">## Utility functions to make the prev and next operations N times</span> <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">prevn</span>(<span class="hljs-params">self, node, n</span>):</span> <span class="hljs-keyword">for</span> _ <span class="hljs-keyword">in</span> range(n): node = node.prev <span class="hljs-keyword">return</span> node <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">nextn</span>(<span class="hljs-params">self, node, n</span>):</span> <span class="hljs-keyword">for</span> _ <span class="hljs-keyword">in</span> range(n): node = node.next <span class="hljs-keyword">return</span> node</code></pre><p>That's it. We have completed our Indexed Linked List implementation!</p><h2 id="heading-conclusions">Conclusions</h2><p>In this article, we defined a new data structure that allows us to do insertions, deletions, and indexing in <code>O(sqrt(N))</code> where <code>N</code> is the length of the list.</p><p>I didn't want to go into the implementation details and proofs of correctness and time complexities because I wanted the article to be accessible for a wider audience. I know some parts of the code can be obscure right now. But I consider that talking about a problem and sharing the ideas to solve it is very helpful for everybody.</p><p>You can see a Lisp implementation of the data structure along with some insights <a target="_blank" href="https://github.com/josejorgers/indexed-linked-list">here</a></p><p>If you want me to make a more detailed article with all the formulas and demonstrations just let me know. You can react to this article and leave a comment, or you can follow me or @-me on Twitter to talk about this or any other CS-related topic.</p>]]><![CDATA[<p>You know the situation is tricky when you need more than just arrays and dictionaries. That was my case a couple of months ago. I needed an indexed data structure that supported insertion and removal operations as fast as possible.</p><p>With an array, I can get the i-th element really fast (this will be called the <em>index</em> operation. I provide an index <code>i</code> and get the <code>i-th</code> element of the list), no matter what the value of i is. But when it comes to inserting or removing an element from the middle of the array things change a little bit.</p><p>On the other hand, linked lists offer a slower index operation. We need to traverse a long segment of the list to get the element we are looking for. But we can make faster insertions and deletions.</p><p>In this article, I will write about a solution I came up with. This is a "new" data structure that somehow combines arrays and linked lists to get a middle point that suited my requirements. I don't know whether this is a truly novel data structure. Furthermore, I don't think it is an awesome idea. I just used some clever and relatively popular techniques to achieve my goal.</p><p>The next section is about the differences between arrays and linked lists, their respective strengths, and their weaknesses. If you are just interested in the data structure specifics, you can skip it. Then I'll talk about the core idea, and later, I'll give some implementation details.</p><blockquote><p>IMPORTANT! I assume you have some basic knowledge of algorithmic complexity and big-O notation.</p></blockquote><h2 id="heading-arrays-vs-linked-lists">Arrays vs. Linked Lists</h2><p>Arrays are contiguous memory slots to store sequences of data. We arrange some elements in such a way that we keep a reference to every single one of them. For example, if we have an array that contains numbers from 1 to 10, we have a reference to every memory slot that contains each of those ten numbers. That allows us to get any element by the index it has in the array really fast.</p><pre><code class="lang-python"><span class="hljs-comment"># Example of array </span>array = [<span class="hljs-number">1</span>, <span class="hljs-number">2</span>, <span class="hljs-number">3</span>, <span class="hljs-number">4</span>, <span class="hljs-number">5</span>, <span class="hljs-number">6</span>, <span class="hljs-number">7</span>, <span class="hljs-number">8</span>, <span class="hljs-number">9</span>, <span class="hljs-number">10</span>]array[<span class="hljs-number">3</span>]<span class="hljs-comment"># >> 4</span>array[<span class="hljs-number">0</span>]<span class="hljs-comment"># >> 1</span></code></pre><p>We have an implicit mapping from index to element. No matter the value of the index, the operation will always be equally fast. That means that doing <code>array[0]</code> is as fast as doing <code>array[9]</code> because we keep all the references at the same time. That's the most useful feature of arrays, we have access to any position in constant time, no matter the size of the array, nor the position itself.</p><p>This great feature is also the reason we can't insert or delete elements with the same ease. We need to keep elements in contiguous positions and maintain the references to them. When deleting an element, we are creating holes in the middle of the array, and that is not allowed. So we need to fix the references manually and fill the holes. The same happens with insertions, in this case, we need to create the hole first and then, fill it with the new element.</p><p>Thus, to insert/delete an element, we need to rearrange the array. If we have an array with <code>N</code> elements, we'll need to rearrange <code>O(N)</code> of those elements when doing an insertion/deletion. I needed to do it faster.</p><p>Linked lists don't maintain all those references in the same way. They are composed of nodes that store an element but also the reference to the next node. This way we only maintain a direct reference to the first node of the list (called the head), this node has a reference to the second one, the second one has a reference to the third one, and so on. In the case of doubly-linked lists, every node has a reference to both the next and the previous node, and we also keep a direct reference to the tail of the list (the last node).</p><p>Thus, looking for an element in a certain position implies to traverse <code>O(N)</code> nodes in the worst case. The good thing is that we can insert or delete an element without worrying about the holes because we only need to change a few references. But I needed to have access to positions in the lists in a faster way.</p><p>I needed to make faster insertion/deletion (better than <code>O(N)</code>), but yet be able to retrieve the <code>i-th</code> element of the list in less than <code>O(N)</code> operations as well. Is important to note that when I say insertion I mean an operation that receives an element and <strong>a position</strong>, and then inserts the element in that position. The delete operation is defined in the same way.</p><p>In the next section, I'll introduce a data structure that lets us fulfill such requirements.</p><h2 id="heading-an-indexed-linked-list">An Indexed Linked List</h2><p>As we saw in the previous section, the "problem" with arrays is that having all those references make the insertion/deletion slow. But the few references of linked lists are a problem as well, because we need to make a lot of operations to get the element at a given position. Thus, it seems reasonable to think about finding a middle point in the number of references.</p><p>First of all, I need to say that no data structure lets us make all the index, insert, and delete operations in constant time. So, a "perfect world" is out of the discussion. Let's move on.</p><p>I was thinking about having a doubly-linked list with more direct references. Instead of just keeping a reference to the first and the last node, I will keep also references to inner nodes, but how many references would I need? If I maintained just a few of them, the index operation might be slow, but if I maintained too many references, then insertions and deletions would take more time.</p><p>After thinking for a while, I came up with the right number of references: <code>sqrt(N)</code> (here <code>sqrt</code> stands for the square root operation). The idea is to keep a list of references, the length of that list will be <code>sqrt(N)</code>. The other important property is that every referenced node will be at a distance of <code>sqrt(N)</code> from the next node in the reference list.</p><p>For example, suppose we have a doubly-linked list with the numbers <code>1, 2, 3, 4, 5</code>. Then our reference list will contain two references because of <code>2 < sqrt(5) < 3</code>. Also, every node referenced in the list will be at a distance of <code>sqrt(5)</code> of the other node referenced in the list. So a possible list will be one that contains the second and the fourth node. Another possibility would be a list with the third and the fifth nodes. In both cases, the distance between nodes is less than <code>sqrt(5)</code>.</p><blockquote><p>When I say <code>sqrt(N)</code> I mean the largest integer <code>n</code> such that <code>n <= sqrt(N)</code></p></blockquote><p>By keeping such a reference list we can make all the index, insert, and delete operations in <code>O(sqrt(N))</code>. Note that we got a slower index operation to achieve a faster insertion/deletion.</p><blockquote><p>By the way, I borrowed some ideas from a technique called <strong>sqrt decomposition</strong>. You can check it out <a target="_blank" href="https://cp-algorithms.com/data_structures/sqrt_decomposition.html">here</a>.</p></blockquote><p>Now I'm going to explain the insert, delete, and index operations.</p><h2 id="heading-inserting-an-element">Inserting an element</h2><p>As I said, I maintain a list of selected nodes. The length of this list is always <code>sqrt(N)</code> and the distance between two consecutive selected nodes in the actual list will be also <code>sqrt(N)</code>.</p><p>Every time we insert a new node in the list we need to check whether the new length of the list has a new integer <code>sqrt(N)</code>. In that case, we need to update the selected nodes list. It would be like:</p><pre><code class="lang-python"><span class="hljs-keyword">import</span> math<span class="hljs-class"><span class="hljs-keyword">class</span> <span class="hljs-title">IndexedLinkedListNode</span>:</span> <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">__init__</span>(<span class="hljs-params">self, content, prev=None, next=None</span>):</span> self.content = content self.prev = prev self.next = next <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">__eq__</span>(<span class="hljs-params">self, other</span>):</span> <span class="hljs-keyword">return</span> self.content == other.content<span class="hljs-class"><span class="hljs-keyword">class</span> <span class="hljs-title">IndexedLinkedList</span>:</span> <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">__init__</span>(<span class="hljs-params">self, elements=[]</span>):</span> self.selected_nodes = [] self.length = <span class="hljs-number">0</span> self.head = <span class="hljs-literal">None</span> self.tail = <span class="hljs-literal">None</span> <span class="hljs-keyword">for</span> e <span class="hljs-keyword">in</span> elements: self.insert(e) <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">insert</span>(<span class="hljs-params">self, element</span>):</span> <span class="hljs-string">''' Inserts an element at the end of the list '''</span> new_node = IndexedLinkedListNode(element) sqlen = math.floor(math.sqrt(self.length)) <span class="hljs-keyword">if</span> self.length == <span class="hljs-number">0</span>: self.selected_nodes.append(new_node) self.head = new_node self.tail = new_node <span class="hljs-keyword">else</span>: self.tail.next = new_node new_node.prev = self.tail self.tail = new_node self.length += <span class="hljs-number">1</span> <span class="hljs-keyword">if</span> math.floor(math.sqrt(self.length)) > sqlen: self.update_list_cause_insertion(new_node) <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">update_list_cause_insertion</span>(<span class="hljs-params">self, new_node</span>):</span> <span class="hljs-string">''' Here we make every selected node become its next node in the list and add the new node to the list '''</span> self.selected_nodes = list(map(<span class="hljs-keyword">lambda</span> node : node.next, self.selected_nodes)) self.selected_nodes.append(new_node) <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">insert_at</span>(<span class="hljs-params">self, element, index</span>):</span> <span class="hljs-string">''' Here we insert the new node before the node that is in the specified position. You can implement yourself a very similar method to insert the new element AFTER the specified position. '''</span> new_node = IndexedLinkedListNode(element) target = self[index] sqlen = math.floor(math.sqrt(self.length)) self.length += <span class="hljs-number">1</span> <span class="hljs-keyword">if</span> index == <span class="hljs-number">0</span>: self.head.prev = new_node new_node.next = self.head self.head = new_node <span class="hljs-keyword">else</span>: target.prev.next = new_node new_node.prev = target.prev new_node.next = target target.prev = new_node <span class="hljs-keyword">if</span> math.floor(math.sqrt(self.length)) > sqlen: self.update_list_cause_insertion(self.tail)</code></pre><p>It can be proved that we always obtain a selected nodes list of length <code>sqrt(N)</code> and with at most <code>sqrt(N)</code> distance between any pair of consecutive nodes. I'm going to skip the demonstration to keep the article as simple as possible.</p><h2 id="heading-deletion">Deletion</h2><p>This is a very similar method. Is like doing the inverse operation. We just need to take care of some corner cases like deleting the only node in the list and deleting the first or the last node.</p><pre><code class="lang-python">... <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">remove_at</span>(<span class="hljs-params">self, index</span>):</span> target = self[index] sqlen = math.floor(math.sqrt(self.length)) selected_pos = <span class="hljs-literal">None</span> <span class="hljs-keyword">try</span>: selected_pos = self.selected_nodes.index(target) <span class="hljs-keyword">except</span>: <span class="hljs-keyword">pass</span> <span class="hljs-keyword">if</span> self.length <= <span class="hljs-number">1</span>: self.head = <span class="hljs-literal">None</span> self.tail = <span class="hljs-literal">None</span> self.selected_nodes = [] self.length = max(<span class="hljs-number">0</span>, self.length - <span class="hljs-number">1</span>) <span class="hljs-keyword">return</span> <span class="hljs-keyword">if</span> target.prev: target.prev.next = target.next <span class="hljs-keyword">else</span>: self.head = target.next <span class="hljs-keyword">if</span> target.next: target.next.prev = target.prev <span class="hljs-keyword">else</span>: self.tail = target.prev self.length -= <span class="hljs-number">1</span> <span class="hljs-keyword">if</span> selected_pos: <span class="hljs-keyword">for</span> selected <span class="hljs-keyword">in</span> self.selected_nodes[selected_pos:]: selected = selected.next <span class="hljs-keyword">if</span> math.floor(math.sqrt(self.length)) < sqlen: self.update_list_cause_removal() <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">update_list_cause_removal</span>(<span class="hljs-params">self</span>):</span> <span class="hljs-string">''' Here we make every selected node become its prev node in the list and remove the last selected node '''</span> self.selected_nodes = self.selected_nodes[:<span class="hljs-number">-1</span>] self.selected_nodes = list(map(<span class="hljs-keyword">lambda</span> node : node.prev, self.selected_nodes))...</code></pre><p>Until here we have achieved the requirements for insertion and deletion methods. But that is assuming that the index operation (<code>self[index]</code>) could be done also in the required time.</p><h2 id="heading-index">Index</h2><p>Let's see how to implement the index operation. This is the last piece of the puzzle. It is a trickier method that would require more explanation to understand the insights. As I said, I would like to keep this article as simple as possible so I'm not going to explain too many details about this operation.</p><pre><code class="lang-python">... <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">__getitem__</span>(<span class="hljs-params">self, index</span>):</span> <span class="hljs-keyword">if</span> len(self.selected_nodes) == <span class="hljs-number">0</span> <span class="hljs-keyword">or</span> index < <span class="hljs-number">0</span> <span class="hljs-keyword">or</span> index >= self.length: <span class="hljs-keyword">return</span> <span class="hljs-literal">None</span> first_index = math.floor(math.sqrt(self.length)) - <span class="hljs-number">1</span> <span class="hljs-comment"># index of the first selected node</span> first_node = self.selected_nodes[<span class="hljs-number">0</span>] <span class="hljs-keyword">if</span> index <= first_index: <span class="hljs-keyword">return</span> self.prevn(first_node, first_index - index) aux_idx = index - first_index target_k = self.integer_positive_root(aux_idx) target_index = target_k**<span class="hljs-number">2</span> + target_k + first_index index -= target_index <span class="hljs-keyword">return</span> self.nextn(self.selected_nodes[target_k], index) <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">integer_positive_root</span>(<span class="hljs-params">self, index</span>):</span> <span class="hljs-string">''' Solves the equation K^2 + K - index '''</span> <span class="hljs-keyword">return</span> math.floor((<span class="hljs-number">-1</span> + math.sqrt(<span class="hljs-number">1</span> + <span class="hljs-number">4</span>*index))/<span class="hljs-number">2</span>) <span class="hljs-comment">## Utility functions to make the prev and next operations N times</span> <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">prevn</span>(<span class="hljs-params">self, node, n</span>):</span> <span class="hljs-keyword">for</span> _ <span class="hljs-keyword">in</span> range(n): node = node.prev <span class="hljs-keyword">return</span> node <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">nextn</span>(<span class="hljs-params">self, node, n</span>):</span> <span class="hljs-keyword">for</span> _ <span class="hljs-keyword">in</span> range(n): node = node.next <span class="hljs-keyword">return</span> node</code></pre><p>That's it. We have completed our Indexed Linked List implementation!</p><h2 id="heading-conclusions">Conclusions</h2><p>In this article, we defined a new data structure that allows us to do insertions, deletions, and indexing in <code>O(sqrt(N))</code> where <code>N</code> is the length of the list.</p><p>I didn't want to go into the implementation details and proofs of correctness and time complexities because I wanted the article to be accessible for a wider audience. I know some parts of the code can be obscure right now. But I consider that talking about a problem and sharing the ideas to solve it is very helpful for everybody.</p><p>You can see a Lisp implementation of the data structure along with some insights <a target="_blank" href="https://github.com/josejorgers/indexed-linked-list">here</a></p><p>If you want me to make a more detailed article with all the formulas and demonstrations just let me know. You can react to this article and leave a comment, or you can follow me or @-me on Twitter to talk about this or any other CS-related topic.</p>]]><![CDATA[Why do I write?]]>https://blog.jayjaydev.com/why-do-i-writehttps://blog.jayjaydev.com/why-do-i-writeTue, 20 Oct 2020 21:26:15 GMT<![CDATA[<p>After my first month of writing in my blog, this seems an adequate question. Why do I write? But also, why now? and what's the benefit I get from it? I started to ask myself these and other related questions about writing. This post is the result of that monologue.</p><h2 id="i-write-to-myself">I write to myself</h2><p>I'd love to have tons of people reading my blog and liking my content. The only social media I'm actively using right now is Twitter. I only use it to get in touch with the community of developers and computer scientists out there. I only tweet about those topics and it'd be amazing to have a lot of followers to debate with. Yes, having a strong online presence is amazing, but you already knew that. The thing is, I don't write for you, nor anyone but me. I really appreciate you writing a comment on any of my posts, and liking them, and following me on Twitter too. Furthermore, I'm very interested in your opinion and feedback and want everyone can enjoy what I'm writing. But I write to myself.</p><p>Writing to myself means I'm not going to write about something I'm not interested in. I know that I could write something about a more popular subject and maybe that way I could get more followers and reactions. That's not my goal, at least right now. I really love to write, it's an amazing learning exercise, I could be helping others which is an amazing feeling, and I'm storing my thoughts for my future self at the same time, which could be an amazing advantage for him. But if I begin to take care of what is popular and what isn't, I'd be losing all the fun.</p><p>I try to write about different things, but always being consistent with why I think is interesting. Notice that doesn't imply that I just write about topics inside my comfort zone, it just means I won't write about what I think is not interesting.</p><p>But why did I begin to write now?</p><h2 id="why-now">Why now?</h2><p>I must recognize the main reason was the pandemic situation. I love to share my knowledge and to discuss with other people about Computer Science, Tech, and Software Engineering. Entering a classroom and spending 90 minutes teaching is almost a therapeutical exercise for me. With the pandemic shut down, I got far from the University and the classrooms. In the first months, I was finishing my thesis and doing some work. But eventually, I started feeling nostalgic about my routine. So I began to use Twitter actively (I made my account in 2018 or 2019 but I rarely used it) and started to have great debates with people from the community. I must say the Twitter community of devs and computer scientists is amazing!</p><p>But I wanted to post differently. I admire people that can make those great, visually attractive, and interesting threads. I try it sometimes, it takes a lot of effort, at least to me. Is easier for me to write an article without taking care of the number of characters.</p><p>I just wanted to care about writing. Even my blog is the most minimalistic Hashnode blog you can do. Sometimes I feel curious about what would happen if I add a picture or some styles. It would be interesting if that really increase the impact of my posts, but I'm lazy enough to don't do it.</p><p>So it's all about the content. But what I have written so far?</p><h2 id="the-topics-i-have-written-about-so-far">The topics I have written about so far</h2><p>My <a target="_blank" href="https://jj.hashnode.dev/the-test-the-bug-and-the-hacker">first article</a> was a motivational one. It was about the necessity of testing and the right way to do it. I also wrote about testing <a target="_blank" href="https://jj.hashnode.dev/building-an-e2e-test-framework-using-some-good-design-patterns">here</a> but making emphasis on some Design Patterns I have used to build testing frameworks.</p><p>I wrote about algorithms and problem-solving strategies <a target="_blank" href="https://jj.hashnode.dev/combining-good-ideas-to-get-the-best-solution">here</a> and <a target="_blank" href="https://jj.hashnode.dev/algorithms-when-can-i-tell-theyre-good-enough">here</a>. Those are two of my favorite topics.</p><p>I don't call myself a developer but I have been earning some money by making software for a while. For those who are starting their developer journey, maybe you can encounter some entertainment and advice in <a target="_blank" href="https://jj.hashnode.dev/my-first-year-as-a-developer">this article</a>.</p><p>I have written about how to make <a target="_blank" href="https://jj.hashnode.dev/make-your-message-stronger-with-better-slides">amazing slides</a> and my thoughts on [competitive programming] (https://jj.hashnode.dev/try-competitive-programming-out).</p><p>My last article was really special. While I was writing it, I was telling myself: "nobody will ever care about this!". But as I said, I find it interesting and I really wanted to write about it. So, that's pretty enough.</p><p>Then I decided to check whether people will really like it or not. I said the Discord server I was thinking of writing more about the topic (which is true). Then, I asked people to read and tell me whether the article seems interesting to them (this is almost a trick because I'm going to write the articles anyway). The result was nice. The post received a lot of reactions (compared with the previous posts, more than 20 reactions are a lot for me) and some acceptance on social media. You can read it <a target="_blank" href="https://jj.hashnode.dev/learn-the-true-polymorphism">here</a>. </p><p>I have been writing at a rate of two articles per week. Yes, I found interesting the 2 articles per week challenge and I accomplished it, but I think I missed something because I haven't earned my badge yet. It's not important anyway, I just wanted to achieve consistency.</p><p>So, what's next?</p><h2 id="next-articles">Next articles</h2><p>I think to write more about Lisp. Not about the language but the paradigm it promotes.</p><p>I have been thinking of writing about Probabilities and Statistics. Those are the subjects I teach. There are a lot of awesome things to talk about but I need to sort them out. It would be nice to include some Artificial Intelligence and Machine Learning as well. I have prepared some exercises and projects for the students, but I need to think a little bit to make articles about this great topic.</p><p>I really want to write more about theoretical topics like computability, algorithmic complexity, optimization, etc. Although I write for myself, I really want to make my content attractive to a lot of people. That's difficult to achieve when talking about those topics. I'd love to write about my research too, but I need to be cautious about future problems when trying to publish in scientific journals.</p><p>And we always need to make room for future, unforeseen experiences and inspirational thoughts!</p><h2 id="conclusions">Conclusions</h2><p>What have I got from writing so far? Well, I have learned a lot about what I have written and about myself. I have received feedback from very talented people and we've had great debates about the articles. I have also improved my English!</p><p>More than 1K people have read at least one paragraph of my content in this first month. That's mind-blowing for me. Really! I wonder whether it would be 2K if I add pictures and styles (bad joke). Is really nice to think I might help at least one fraction of those people.</p><p>And all that while staying consistent with my idea of what an interesting topic is! Great experience.</p><p>As I always say, leave your comment and/or reaction, I'd be glad to receive it. You also can follow me on Twitter, the link is at the top of the page. Stay tuned if you liked this one! I plan to post periodically.</p>]]><![CDATA[<p>After my first month of writing in my blog, this seems an adequate question. Why do I write? But also, why now? and what's the benefit I get from it? I started to ask myself these and other related questions about writing. This post is the result of that monologue.</p><h2 id="i-write-to-myself">I write to myself</h2><p>I'd love to have tons of people reading my blog and liking my content. The only social media I'm actively using right now is Twitter. I only use it to get in touch with the community of developers and computer scientists out there. I only tweet about those topics and it'd be amazing to have a lot of followers to debate with. Yes, having a strong online presence is amazing, but you already knew that. The thing is, I don't write for you, nor anyone but me. I really appreciate you writing a comment on any of my posts, and liking them, and following me on Twitter too. Furthermore, I'm very interested in your opinion and feedback and want everyone can enjoy what I'm writing. But I write to myself.</p><p>Writing to myself means I'm not going to write about something I'm not interested in. I know that I could write something about a more popular subject and maybe that way I could get more followers and reactions. That's not my goal, at least right now. I really love to write, it's an amazing learning exercise, I could be helping others which is an amazing feeling, and I'm storing my thoughts for my future self at the same time, which could be an amazing advantage for him. But if I begin to take care of what is popular and what isn't, I'd be losing all the fun.</p><p>I try to write about different things, but always being consistent with why I think is interesting. Notice that doesn't imply that I just write about topics inside my comfort zone, it just means I won't write about what I think is not interesting.</p><p>But why did I begin to write now?</p><h2 id="why-now">Why now?</h2><p>I must recognize the main reason was the pandemic situation. I love to share my knowledge and to discuss with other people about Computer Science, Tech, and Software Engineering. Entering a classroom and spending 90 minutes teaching is almost a therapeutical exercise for me. With the pandemic shut down, I got far from the University and the classrooms. In the first months, I was finishing my thesis and doing some work. But eventually, I started feeling nostalgic about my routine. So I began to use Twitter actively (I made my account in 2018 or 2019 but I rarely used it) and started to have great debates with people from the community. I must say the Twitter community of devs and computer scientists is amazing!</p><p>But I wanted to post differently. I admire people that can make those great, visually attractive, and interesting threads. I try it sometimes, it takes a lot of effort, at least to me. Is easier for me to write an article without taking care of the number of characters.</p><p>I just wanted to care about writing. Even my blog is the most minimalistic Hashnode blog you can do. Sometimes I feel curious about what would happen if I add a picture or some styles. It would be interesting if that really increase the impact of my posts, but I'm lazy enough to don't do it.</p><p>So it's all about the content. But what I have written so far?</p><h2 id="the-topics-i-have-written-about-so-far">The topics I have written about so far</h2><p>My <a target="_blank" href="https://jj.hashnode.dev/the-test-the-bug-and-the-hacker">first article</a> was a motivational one. It was about the necessity of testing and the right way to do it. I also wrote about testing <a target="_blank" href="https://jj.hashnode.dev/building-an-e2e-test-framework-using-some-good-design-patterns">here</a> but making emphasis on some Design Patterns I have used to build testing frameworks.</p><p>I wrote about algorithms and problem-solving strategies <a target="_blank" href="https://jj.hashnode.dev/combining-good-ideas-to-get-the-best-solution">here</a> and <a target="_blank" href="https://jj.hashnode.dev/algorithms-when-can-i-tell-theyre-good-enough">here</a>. Those are two of my favorite topics.</p><p>I don't call myself a developer but I have been earning some money by making software for a while. For those who are starting their developer journey, maybe you can encounter some entertainment and advice in <a target="_blank" href="https://jj.hashnode.dev/my-first-year-as-a-developer">this article</a>.</p><p>I have written about how to make <a target="_blank" href="https://jj.hashnode.dev/make-your-message-stronger-with-better-slides">amazing slides</a> and my thoughts on [competitive programming] (https://jj.hashnode.dev/try-competitive-programming-out).</p><p>My last article was really special. While I was writing it, I was telling myself: "nobody will ever care about this!". But as I said, I find it interesting and I really wanted to write about it. So, that's pretty enough.</p><p>Then I decided to check whether people will really like it or not. I said the Discord server I was thinking of writing more about the topic (which is true). Then, I asked people to read and tell me whether the article seems interesting to them (this is almost a trick because I'm going to write the articles anyway). The result was nice. The post received a lot of reactions (compared with the previous posts, more than 20 reactions are a lot for me) and some acceptance on social media. You can read it <a target="_blank" href="https://jj.hashnode.dev/learn-the-true-polymorphism">here</a>. </p><p>I have been writing at a rate of two articles per week. Yes, I found interesting the 2 articles per week challenge and I accomplished it, but I think I missed something because I haven't earned my badge yet. It's not important anyway, I just wanted to achieve consistency.</p><p>So, what's next?</p><h2 id="next-articles">Next articles</h2><p>I think to write more about Lisp. Not about the language but the paradigm it promotes.</p><p>I have been thinking of writing about Probabilities and Statistics. Those are the subjects I teach. There are a lot of awesome things to talk about but I need to sort them out. It would be nice to include some Artificial Intelligence and Machine Learning as well. I have prepared some exercises and projects for the students, but I need to think a little bit to make articles about this great topic.</p><p>I really want to write more about theoretical topics like computability, algorithmic complexity, optimization, etc. Although I write for myself, I really want to make my content attractive to a lot of people. That's difficult to achieve when talking about those topics. I'd love to write about my research too, but I need to be cautious about future problems when trying to publish in scientific journals.</p><p>And we always need to make room for future, unforeseen experiences and inspirational thoughts!</p><h2 id="conclusions">Conclusions</h2><p>What have I got from writing so far? Well, I have learned a lot about what I have written and about myself. I have received feedback from very talented people and we've had great debates about the articles. I have also improved my English!</p><p>More than 1K people have read at least one paragraph of my content in this first month. That's mind-blowing for me. Really! I wonder whether it would be 2K if I add pictures and styles (bad joke). Is really nice to think I might help at least one fraction of those people.</p><p>And all that while staying consistent with my idea of what an interesting topic is! Great experience.</p><p>As I always say, leave your comment and/or reaction, I'd be glad to receive it. You also can follow me on Twitter, the link is at the top of the page. Stay tuned if you liked this one! I plan to post periodically.</p>]]><![CDATA[Learn the true polymorphism]]>https://blog.jayjaydev.com/learn-the-true-polymorphismhttps://blog.jayjaydev.com/learn-the-true-polymorphismSun, 18 Oct 2020 05:15:08 GMT<![CDATA[<p>Polymorphism is the ability of an object to take on many forms. It is an everyday word in Object-Oriented Programming (OOP). We have classes that inherit from other classes. We can treat an instance of a child class as an instance of the parent class, but it will behave differently.</p><p>Common OOP languages have a very limited version of polymorphism. In this post, I'm going to talk about a wider version of polymorphism that is included in Lisp.</p><h2 id="a-first-example">A first example</h2><p>Let's see first what can be done with the polymorphism we commonly use.</p><p>As a simple example, we can have a class called <code>Shape</code> with a method called <code>Area</code>. We also can have other classes called <code>Triangle</code>, <code>Rectangle</code>, <code>Circle</code>, etc. Those classes inherit from <code>Shape</code>, and each one provides its own implementation of the <code>Area</code> method.</p><p>We can make a list that only contains shape instances and treats every element as an instance of the <code>Shape</code> class. But when we do <code>element.Area()</code> we get different implementations every time. No matter that we treat every element as a <code>Shape</code> instance, every one of them behaves as a <code>Triangle</code>, a <code>Circle</code>, or a <code>Rectangle</code> according to its class.</p><p>That's the kind of polymorphism you might use in your favorite OOP language. But what if I tell you that's a very limited kind of polymorphism?</p><h2 id="another-example">Another example</h2><p>In the previous example, we had a method inside a class that got overridden by every child class. That was good because we wanted to have different implementations of the <code>Area</code> method for every shape.</p><p>Let's think about another example. Now we have a class <code>Drum</code>, and also let's create the class <code>Stick</code>. Where do you include the <code>Play</code> method? Inside the <code>Drum</code> class? Inside the <code>Stick</code> class?</p><p>If you choose to include it inside <code>Drum</code>, then you can have different implementations of <code>Play</code> for every different drum you create. But what happens when you have a different stick? You can't have polymorphism with respect to the <code>Stick</code> instance! Exactly the opposite happens if you decide to include the method inside the <code>Stick</code> class: you'll have polymorphism on the <code>Stick</code> instances but not on the drums.</p><p>Where's your polymorphism now?!</p><h2 id="taking-a-deeper-look">Taking a deeper look</h2><p>Let's think about this issue. What happens when you do <code>classInstance.ClassMethod(<args>)</code>?</p><p>We can say we are invoking the <code>ClassMethod</code> inside the <code>classInstance</code>. But let's change the previous code a little bit:</p><pre><code class="lang-pseudocode"> ClassMethod(classInstance, <args>)</code></pre><p>Now we see things slightly different. We are invoking the <code>ClassMethod</code> passing the <code>classInstance</code> as the first argument.</p><p>Maybe this is familiar to you. For example, in Python, you can invoke class methods using both ways interchangeably. Both of them are equivalent.</p><p>Using the second way, we can achieve different <code>ClassMethod</code> behavior by changing the <code>classInstance</code> argument. That's equivalent to polymorphism we use every day. That's called <em>polymorphism in the first argument</em>.</p><p>Let's return to our "drum and stick" example. The solution would be having something like this:</p><pre><code class="lang-pseudocode">Play(drumInstance, stickInstance, <args>)</code></pre><p>Now we can have different behavior for every Drum-Stick combination. Nice!</p><p>But that's not a natural solution in many programming languages. Usually, every method has the "special" first parameter reserved for the instance of the class where it is defined. Hence, we can't have many "special" parameters representing many different instances of different classes. Even more, many OOP languages don't allow you to define methods outside of a class. So yes, you probably have been using a very limited version of polymorphism during your coding life.</p><p>Let's see what Lisp has to say!</p><h2 id="the-lisp-solution">The Lisp solution</h2><p>Lisp is a very old programming language. It's so old that there was no OOP when it was created. But in the '80s a new update of the language came up with the Common Lisp Object System (CLOS), And then, Lisp became an OOP language.</p><blockquote><p>As a matter of fact, CLOS is implemented in Lisp itself. There are a lot of great things about Lisp. I will definitely write about more of those things shortly.</p></blockquote><p>There are no class methods in Lisp. Methods don't belong to the class. You have classes with properties and methods that can receive instances of those classes as arguments. It's the solution we talked about! That's how Lisp provides us with <em>polymorphism on multiple arguments</em>.</p><p>Of course, you can solve all kinds of problems with the "regular" polymorphism. You also can solve all kinds of problems without OOP at all! You can think about solving the Drum-Stick problem with a design pattern or some abstraction. But polymorphism in multiple arguments is explicit and simple.</p><p>Lisp's CLOS has a lot of more interesting features. This was the first one that really impressed me. Also was the first one I exploited a lot for solving some problems. Talking about other features of CLOS requires some prior knowledge of the language, but I plan to write about topics that don't demand such knowledge. Like this one!</p><p>Let's summarize what we have been talking about.</p><h2 id="conclusions">Conclusions</h2><p>Polymorphism is a cool feature in OOP. It is the ability of classes to behave differently, in different scenarios. But the kind of polymorphism that is present in popular OOP languages is limited.</p><p>Yes, there are different types of polymorphisms. We commonly use <em>polymorphism on the first argument</em>, but we can have <em>polymorphism on multiple arguments</em>. To achieve that, we need to separate methods from classes.</p><p>Lisp is a multiparadigm (Object-Oriented included) programming language. In Lisp, methods are created outside classes, and we can have polymorphism on multiple arguments.</p><p>Using this more general polymorphism you can get rid of some abstractions and design patterns that maybe you have been using to solve problems similar to the Drum-Stick one.</p><p>There are a lot of other features about Lisp that are mind-blowing. It really changes the way you think about programming. I plan to write about some of those features with a focus on the change of paradigm instead of the language.</p><p>But I recommend you to take a look at Lisp, and to experience those features and different ways of thinking by yourself.</p><p>If you liked the post let me know. You can leave a comment with whatever you want to say. You can also follow me on Twitter for debates and thoughts about related topics.</p>]]><![CDATA[<p>Polymorphism is the ability of an object to take on many forms. It is an everyday word in Object-Oriented Programming (OOP). We have classes that inherit from other classes. We can treat an instance of a child class as an instance of the parent class, but it will behave differently.</p><p>Common OOP languages have a very limited version of polymorphism. In this post, I'm going to talk about a wider version of polymorphism that is included in Lisp.</p><h2 id="a-first-example">A first example</h2><p>Let's see first what can be done with the polymorphism we commonly use.</p><p>As a simple example, we can have a class called <code>Shape</code> with a method called <code>Area</code>. We also can have other classes called <code>Triangle</code>, <code>Rectangle</code>, <code>Circle</code>, etc. Those classes inherit from <code>Shape</code>, and each one provides its own implementation of the <code>Area</code> method.</p><p>We can make a list that only contains shape instances and treats every element as an instance of the <code>Shape</code> class. But when we do <code>element.Area()</code> we get different implementations every time. No matter that we treat every element as a <code>Shape</code> instance, every one of them behaves as a <code>Triangle</code>, a <code>Circle</code>, or a <code>Rectangle</code> according to its class.</p><p>That's the kind of polymorphism you might use in your favorite OOP language. But what if I tell you that's a very limited kind of polymorphism?</p><h2 id="another-example">Another example</h2><p>In the previous example, we had a method inside a class that got overridden by every child class. That was good because we wanted to have different implementations of the <code>Area</code> method for every shape.</p><p>Let's think about another example. Now we have a class <code>Drum</code>, and also let's create the class <code>Stick</code>. Where do you include the <code>Play</code> method? Inside the <code>Drum</code> class? Inside the <code>Stick</code> class?</p><p>If you choose to include it inside <code>Drum</code>, then you can have different implementations of <code>Play</code> for every different drum you create. But what happens when you have a different stick? You can't have polymorphism with respect to the <code>Stick</code> instance! Exactly the opposite happens if you decide to include the method inside the <code>Stick</code> class: you'll have polymorphism on the <code>Stick</code> instances but not on the drums.</p><p>Where's your polymorphism now?!</p><h2 id="taking-a-deeper-look">Taking a deeper look</h2><p>Let's think about this issue. What happens when you do <code>classInstance.ClassMethod(<args>)</code>?</p><p>We can say we are invoking the <code>ClassMethod</code> inside the <code>classInstance</code>. But let's change the previous code a little bit:</p><pre><code class="lang-pseudocode"> ClassMethod(classInstance, <args>)</code></pre><p>Now we see things slightly different. We are invoking the <code>ClassMethod</code> passing the <code>classInstance</code> as the first argument.</p><p>Maybe this is familiar to you. For example, in Python, you can invoke class methods using both ways interchangeably. Both of them are equivalent.</p><p>Using the second way, we can achieve different <code>ClassMethod</code> behavior by changing the <code>classInstance</code> argument. That's equivalent to polymorphism we use every day. That's called <em>polymorphism in the first argument</em>.</p><p>Let's return to our "drum and stick" example. The solution would be having something like this:</p><pre><code class="lang-pseudocode">Play(drumInstance, stickInstance, <args>)</code></pre><p>Now we can have different behavior for every Drum-Stick combination. Nice!</p><p>But that's not a natural solution in many programming languages. Usually, every method has the "special" first parameter reserved for the instance of the class where it is defined. Hence, we can't have many "special" parameters representing many different instances of different classes. Even more, many OOP languages don't allow you to define methods outside of a class. So yes, you probably have been using a very limited version of polymorphism during your coding life.</p><p>Let's see what Lisp has to say!</p><h2 id="the-lisp-solution">The Lisp solution</h2><p>Lisp is a very old programming language. It's so old that there was no OOP when it was created. But in the '80s a new update of the language came up with the Common Lisp Object System (CLOS), And then, Lisp became an OOP language.</p><blockquote><p>As a matter of fact, CLOS is implemented in Lisp itself. There are a lot of great things about Lisp. I will definitely write about more of those things shortly.</p></blockquote><p>There are no class methods in Lisp. Methods don't belong to the class. You have classes with properties and methods that can receive instances of those classes as arguments. It's the solution we talked about! That's how Lisp provides us with <em>polymorphism on multiple arguments</em>.</p><p>Of course, you can solve all kinds of problems with the "regular" polymorphism. You also can solve all kinds of problems without OOP at all! You can think about solving the Drum-Stick problem with a design pattern or some abstraction. But polymorphism in multiple arguments is explicit and simple.</p><p>Lisp's CLOS has a lot of more interesting features. This was the first one that really impressed me. Also was the first one I exploited a lot for solving some problems. Talking about other features of CLOS requires some prior knowledge of the language, but I plan to write about topics that don't demand such knowledge. Like this one!</p><p>Let's summarize what we have been talking about.</p><h2 id="conclusions">Conclusions</h2><p>Polymorphism is a cool feature in OOP. It is the ability of classes to behave differently, in different scenarios. But the kind of polymorphism that is present in popular OOP languages is limited.</p><p>Yes, there are different types of polymorphisms. We commonly use <em>polymorphism on the first argument</em>, but we can have <em>polymorphism on multiple arguments</em>. To achieve that, we need to separate methods from classes.</p><p>Lisp is a multiparadigm (Object-Oriented included) programming language. In Lisp, methods are created outside classes, and we can have polymorphism on multiple arguments.</p><p>Using this more general polymorphism you can get rid of some abstractions and design patterns that maybe you have been using to solve problems similar to the Drum-Stick one.</p><p>There are a lot of other features about Lisp that are mind-blowing. It really changes the way you think about programming. I plan to write about some of those features with a focus on the change of paradigm instead of the language.</p><p>But I recommend you to take a look at Lisp, and to experience those features and different ways of thinking by yourself.</p><p>If you liked the post let me know. You can leave a comment with whatever you want to say. You can also follow me on Twitter for debates and thoughts about related topics.</p>]]><![CDATA[Try Competitive Programming out!]]>https://blog.jayjaydev.com/try-competitive-programming-outhttps://blog.jayjaydev.com/try-competitive-programming-outSat, 17 Oct 2020 13:49:55 GMT<![CDATA[<p>International Collegiate Programming Contest (ICPC) is a competition where teams of students try to solve hard problems that involve algorithms and data structures. After being myself a (terrible) contestant, I have my own opinion about what ICPC can give to a student.</p><p>In this post, I will talk about what I think is good about ICPC and some possible drawbacks.</p><h2 id="icpc-training">ICPC Training</h2><p>This is the core activity an ICPC contestant makes. You need to train a lot to classify to a higher instance of the competition (eg: a Regional or a World Final). So the first benefit is this one, you get used to work a lot.</p><p>But the benefits are not limited to the amount of training. The training way is the main feature a contestant can develop.</p><p> Great contestants (not me, although I luckily know a bunch of them) spend entire days solving a lot of challenging problems.</p><p>It's not just learning binary search, it's solving a lot of problems using binary search. I have never seen a better preparation.</p><blockquote><p>By the way, you can solve a lot of awesome problems using Binary Search. I could write a post about it in the future but there is a lot of great material all over the internet.</p></blockquote><p>Think about how many times you, or some of your peers, read about something and right after finish claim to have learned something new. You haven't learned anything until you can apply it to solve several problems. You need to aim to become that new thing in something natural for you.</p><p>That's what a good ICPC contestant does.</p><h2 id="teamwork">Teamwork</h2><p>In ICPC you have two teammates and all of you need to share a single PC. Contest last about 4-5 hrs that are rarely enough. A good team is not just the sum of the individual value of the three members. Teamwork and partnership can play an important role.</p><p>So you don't only train yourself, you need to make training sessions with the rest of the team, sharing a single PC. You need to explain to others your reasoning and debate with them your solutions. </p><p>Here the only issue is that you get used to communicate with people that think pretty much like you. Then, when you need to talk to someone that is not a contestant about your work, it can be tricky.</p><p>But the important thing here is that you get used to share resources and respect the work of your peers.</p><p>In our society, this skill is priceless.</p><h2 id="what-about-real-life">What about real life?</h2><p>Problems you solve in ICPC are well-posed and you know that a bunch of people has solved it. If you become a researcher tomorrow you'll find real life is far different.</p><p>You only make little console applications, only use C++ because is faster, don't use AI, nor ML, nor Design Patterns. If you've only coded ICPC problems you are a terrible developer right now. You don't have the most important skills a regular company is looking for.</p><p>There are a lot of choices out there to sharp whatever skill you want. You can dedicate part of your time to learn Object-Oriented Programming or ML. You can contribute to an Open Source project and do some research. It would be a shame you only make one activity. If you have good possibilities to achieve awesome results in ICPC, train a lot. But you also can train a little bit and make other activities with the rest of your time.</p><p>I see ICPC training as a way to shape your learning process. But the great thing about it is that you actually learn a lot of awesome techniques by training! And you really learn them, you won't forget them tomorrow (if you did it right).</p><p>When a contestant poorly performs a real-life task, that's on the contestant, not on the contest. Training competitive programming doesn't erase the rest of your capabilities, but taking care of them is up to you.</p><p>I know World Finals competitors that are good in other activities like AI. They spent months training a lot, they were pros. Now they are a great choice for any company or research department.</p><h2 id="conclusions">Conclusions</h2><p>I have heard a lot of different opinions about the about Competitive Programming. I really don't think that activity harms you in any way.</p><p>Yes, you need to be aware that the outside world is different. Is up to you to stay tuned with real life.</p><p>Being a contestant allowed me to know very talented people (that's another benefit!). I don't know if I would like to have trained harder, but I'm sure I'm better because of my training.</p><p>Of course, I recommend you to try competitive programming out. If you are a University student you can even get involved in ICPC competitions. But you can always try to solve one or two competitive programming problems each week. I assure you it doesn't hurt.</p><p>Feel free to leave your comment with any impression, doubt, or suggestion. You can follow me on Twitter for more talks about related topics.</p>]]><![CDATA[<p>International Collegiate Programming Contest (ICPC) is a competition where teams of students try to solve hard problems that involve algorithms and data structures. After being myself a (terrible) contestant, I have my own opinion about what ICPC can give to a student.</p><p>In this post, I will talk about what I think is good about ICPC and some possible drawbacks.</p><h2 id="icpc-training">ICPC Training</h2><p>This is the core activity an ICPC contestant makes. You need to train a lot to classify to a higher instance of the competition (eg: a Regional or a World Final). So the first benefit is this one, you get used to work a lot.</p><p>But the benefits are not limited to the amount of training. The training way is the main feature a contestant can develop.</p><p> Great contestants (not me, although I luckily know a bunch of them) spend entire days solving a lot of challenging problems.</p><p>It's not just learning binary search, it's solving a lot of problems using binary search. I have never seen a better preparation.</p><blockquote><p>By the way, you can solve a lot of awesome problems using Binary Search. I could write a post about it in the future but there is a lot of great material all over the internet.</p></blockquote><p>Think about how many times you, or some of your peers, read about something and right after finish claim to have learned something new. You haven't learned anything until you can apply it to solve several problems. You need to aim to become that new thing in something natural for you.</p><p>That's what a good ICPC contestant does.</p><h2 id="teamwork">Teamwork</h2><p>In ICPC you have two teammates and all of you need to share a single PC. Contest last about 4-5 hrs that are rarely enough. A good team is not just the sum of the individual value of the three members. Teamwork and partnership can play an important role.</p><p>So you don't only train yourself, you need to make training sessions with the rest of the team, sharing a single PC. You need to explain to others your reasoning and debate with them your solutions. </p><p>Here the only issue is that you get used to communicate with people that think pretty much like you. Then, when you need to talk to someone that is not a contestant about your work, it can be tricky.</p><p>But the important thing here is that you get used to share resources and respect the work of your peers.</p><p>In our society, this skill is priceless.</p><h2 id="what-about-real-life">What about real life?</h2><p>Problems you solve in ICPC are well-posed and you know that a bunch of people has solved it. If you become a researcher tomorrow you'll find real life is far different.</p><p>You only make little console applications, only use C++ because is faster, don't use AI, nor ML, nor Design Patterns. If you've only coded ICPC problems you are a terrible developer right now. You don't have the most important skills a regular company is looking for.</p><p>There are a lot of choices out there to sharp whatever skill you want. You can dedicate part of your time to learn Object-Oriented Programming or ML. You can contribute to an Open Source project and do some research. It would be a shame you only make one activity. If you have good possibilities to achieve awesome results in ICPC, train a lot. But you also can train a little bit and make other activities with the rest of your time.</p><p>I see ICPC training as a way to shape your learning process. But the great thing about it is that you actually learn a lot of awesome techniques by training! And you really learn them, you won't forget them tomorrow (if you did it right).</p><p>When a contestant poorly performs a real-life task, that's on the contestant, not on the contest. Training competitive programming doesn't erase the rest of your capabilities, but taking care of them is up to you.</p><p>I know World Finals competitors that are good in other activities like AI. They spent months training a lot, they were pros. Now they are a great choice for any company or research department.</p><h2 id="conclusions">Conclusions</h2><p>I have heard a lot of different opinions about the about Competitive Programming. I really don't think that activity harms you in any way.</p><p>Yes, you need to be aware that the outside world is different. Is up to you to stay tuned with real life.</p><p>Being a contestant allowed me to know very talented people (that's another benefit!). I don't know if I would like to have trained harder, but I'm sure I'm better because of my training.</p><p>Of course, I recommend you to try competitive programming out. If you are a University student you can even get involved in ICPC competitions. But you can always try to solve one or two competitive programming problems each week. I assure you it doesn't hurt.</p><p>Feel free to leave your comment with any impression, doubt, or suggestion. You can follow me on Twitter for more talks about related topics.</p>]]><![CDATA[Building an E2E Test Framework using some good Design Patterns]]>https://blog.jayjaydev.com/building-an-e2e-test-framework-using-some-good-design-patternshttps://blog.jayjaydev.com/building-an-e2e-test-framework-using-some-good-design-patternsSat, 10 Oct 2020 05:29:57 GMT<![CDATA[<p>End to End (E2E) testing is about simulating the user experience. It doesn't deal with functions, variables, classes, or databases. Instead, it deals with buttons, clicks, expected messages, links, etc. We can say that E2E testing is the "ultimate" testing since it checks whether the product as a whole behaves as expected.</p><p>In general, E2E testing is difficult to automatize. First of all, we need tools that can interact with the application that is being tested, we need to fill forms, wait for a page to load completely, and that kind of stuff. We also need to get the results from the user interface, we don't have functions returning objects but HTML elements containing the information. Mocking a real user can be challenging and might require a lot of maintenance.</p><p>In this post, I will talk about my own experience building an E2E Testing Framework, I applied some cool Design Patterns so I think this could be interesting for you even if you have nothing to do with E2E Testing Automation.</p><p>This post is <em>language and tool agnostic</em>, it means that I won't refer to a specific programming language nor a specific E2E tool like Selenium, Puppeteer, or Playwright. By the way, those are great tools for automatizing E2E tests. Furthermore, this post focuses on E2E Testing for websites.</p><h2 id="the-problem">The Problem</h2><p>I was required to design a framework to perform different E2E tests on different websites. More precisely, I needed to make some tests over specific React components inside those websites. Every component has the same structure and CSS selectors no matter the website and just changes slightly from one site to another. We needed to make tests for every possible viewport (mobile, tablet, and desktop), and the components change their structure when the viewport changes.</p><p>Furthermore, I know nothing about the developers. Thus, I needed to be prepared to manage some unforeseen changes in the interface relatively easy. In other words, one critical requirement the framework needed to fulfill was to be easy to maintain.</p><p>How to make an E2E Testing Framework that doesn't care too much whether developers changed the id attribute of some button that is clicked in some test? How to be able to write tests for some component that is not created yet? And, if possible, how to make every test easy to read and understand?</p><p>All those goals can be achieved by applying some abstractions and design patterns. Here we go!</p><h2 id="the-page-object-model">The Page Object Model</h2><p>The first thing we need to do is to create an abstraction for a page. This is important for several reasons. It will increase readability. For example, you don't want to have a line in your test that reads <code>tool.getByCssSelector("button.btn.btn-submit").click()</code>, instead you want to have a line like this one: <code>page.clickSubmitLoginFormButton()</code> or something similar. You also need to keep all the CSS selectors and DOM related stuff in a single place, this way, when something in the interface is changed you only need to modify one single file (or maybe two, but not more ;-) ).</p><p>That abstraction is called the <strong>Page Object Model</strong>. You make a class that represents only the elements that you are interested in from the page. You put all the DOM related stuff in those classes.</p><p>In my case, I did it slightly different. I created two classes for every page. A <strong>PageModel</strong> and a <strong>Page Object</strong>. In the first one, I put the elements of the page. For example, suppose we are testing a login page, then my <strong>LoginPageModel</strong> would be like:</p><pre><code class="lang-pseudocode">class LoginPageModel constructor(tool) this.tool = tool loginUsernameInput() return this.tool.getById('username-input') loginPasswordInput() return this.tool.getById('password-input) loginSubmitButton() return this.tool.getById('submit-login-button')</code></pre><p>If any of those elements change in the future we only need to modify the corresponding <strong>PageModel</strong> class.</p><p>In the <strong>PageObject</strong> class, I add the actions that you can perform on the page. An example of a <strong>LoginPageObject</strong> class would be:</p><pre><code class="lang-pseudocode">class LoginPageObject constructor(pageModel) this.model = pageModel typeUsername(username) this.model.loginUsernameInput().type(username) typePassword(password) this.model.loginPasswordInput().type(password) clickLoginSubmitButton() this.model.loginSubmitButton().click()</code></pre><p>Here we can get advantage of a statically typed language that can get all the methods of the model class in compilation time, that way some IntelliSense tool can remind us the name of every method that represents a page element, and we also get more compilation errors and fewer runtime errors, which is very good for us and our mental health.</p><p>Why to separate page elements from page actions? A single class that contains both the elements and the actions can be very large. We can say that by doing this we are applying the Single Responsibility Principle and that would be cool, but in this case, that has not too many practical significance beyond readability and keeping classes simple.</p><p>With the <strong>Page Object</strong> abstraction we can make tests that only depend on page objects instead of writing some tricky CSS selectors in the middle of the test code. We keep all the DOM related stuff in a single place and our tests can be more expressive and easy to understand.</p><h2 id="tests-the-facade-pattern">Tests. The Facade Pattern</h2><p>Now we have many classes that contain all the elements and actions of several pages. What we need to do now is to build our tests. Those tests will provide a simple interface that exposes to the client the <code>run</code> functionality. This functionality returns a test result. The client doesn't have to worry about accessing any element or doing any action, just needs to instantiate the test and run it.</p><p>When we provide a simple interface that hides a more complex infrastructure we are applying the Facade Pattern. I know that's only a fancy name for something that is clear we needed to do. Continuing with our Login Page Test example, the <strong>LoginTest</strong> would be something like this:</p><pre><code class="lang-pseudocode">class LoginTest constructor(loginPageObject) this.pageObject = loginPageObject run() this.pageObject.typeUsername("TestUser") this.pageObject.typePassword("TestPassword") this.pageObject.clickLoginSubmitButton() assert that the login was successful</code></pre><p>The last line of the <code>run</code> method is an assertion. Depending on the complexity of the assertions you use, you can either define them separately or inside the Page Object. By choosing the first option you can reuse and extend assertions, but if your assertions are very specific for each case and simple enough the first option can be overkill and you probably will be good with the second one.</p><p>We are also injecting the Page Object dependency in the Test. We are not doing <code>this.pageObject = new LoginPageObject()</code> but receiving the dependency as an argument in the constructor. This is called <em>Dependency Injection</em>. That way, we can instantiate the same test for another page. We also inject the Page Model in Page Object instances. Then, we can have the same Page Object with another model (eg: same LoginPageObject instance with a LoginMobilePageModel instead of a regular LoginPageModel).</p><p>But now, to instantiate a test, we need to instantiate one or more Page Models, then one or more Page Objects, and finally the Test. It seems too much work. That's precisely one of the drawbacks of using Dependency Injection, but it is solvable!</p><h2 id="the-factory-pattern">The Factory Pattern</h2><p>If it's difficult, let's delegate the responsibility to another abstraction. In this case, we'll make some Factories. These are classes that are used to instantiate other classes. Every Factory class will be responsible for instantiating a specific test. That's the Factory Pattern in action!</p><p>So we can create a <strong>LoginTestFactory</strong> for our LoginTest:</p><pre><code class="lang-pseudocode">import toolclass LoginTestFactory create(config) if config.viewport == 'mobile' then return new LoginTest(new LoginPageObject(new LoginMobilePageModel(tool))) else return new LoginTest(new LoginPageObject(new LoginPageModel(tool)))</code></pre><p>Here we are representing with <code>tool</code> any possible technology you could use to get the elements of a page and interact with them. Maybe you don't pass the imported tool as is, but you create some objects using that tool and then pass those objects as parameters. But the idea is that all the relatively complex logic to make an instance of a Test is encapsulated in a Factory object.</p><p>To run our test we only need to do something like this:</p><pre><code class="lang-pseudocode">runLoginTestDesktop() factory = new LoginTestFactory() config = new ConfigObject(viewport = 'desktop') test = factory.create(config) test.run()runLoginTestMobile() factory = new LoginTestFactory() config = new ConfigObject(viewport = 'mobile') test = factory.create(config) test.run()</code></pre><p>Now, in the Conclusions section, we'll check whether we have accomplished our initial goals</p><h2 id="conclusions">Conclusions</h2><p>Building our framework in the way I have shown you in this post, can dramatically decrease the cost of a change in the user interface. All the code that depends on the user interface is isolated in specific classes that abstract the concept of a page.</p><p>That abstraction also allows us to write the tests for the next week! I mean the test for components that have not been created yet. We just make the required new PageModels and PageObjects to mock the elements on the page that will be created and we can build the rest of the process in the same way we have seen so far. When we had the specific elements on the interface we can change the page models and verify whether the application behaves as expected.</p><p>We also have tests that are very easy to read and understand since we make expressive actions like <code>this.pageObject.clickLoginSubmitButton()</code>. Thus, our tests can describe the requirements of our application and can be easily maintained.</p><p>E2E testing automation is difficult because is difficult to keep it simple, and a complex test is not a test. In this post, I have shown some design patterns and good practices to make it smoother. I have tried to make it language and tool agnostic in order you can apply these practices in your project no matter what language or technology you are using. I only assumed an Object-Oriented programming language.</p><p>No matter if you are not making an E2E Testing Framework, I made this post because I think some of these tricks can be applied in a relatively wide variety of problems.</p><p>Feel free to comment any suggestion, impression, or doubt about this article below. You can also follow me on Twitter for debates about related topics.</p>]]><![CDATA[<p>End to End (E2E) testing is about simulating the user experience. It doesn't deal with functions, variables, classes, or databases. Instead, it deals with buttons, clicks, expected messages, links, etc. We can say that E2E testing is the "ultimate" testing since it checks whether the product as a whole behaves as expected.</p><p>In general, E2E testing is difficult to automatize. First of all, we need tools that can interact with the application that is being tested, we need to fill forms, wait for a page to load completely, and that kind of stuff. We also need to get the results from the user interface, we don't have functions returning objects but HTML elements containing the information. Mocking a real user can be challenging and might require a lot of maintenance.</p><p>In this post, I will talk about my own experience building an E2E Testing Framework, I applied some cool Design Patterns so I think this could be interesting for you even if you have nothing to do with E2E Testing Automation.</p><p>This post is <em>language and tool agnostic</em>, it means that I won't refer to a specific programming language nor a specific E2E tool like Selenium, Puppeteer, or Playwright. By the way, those are great tools for automatizing E2E tests. Furthermore, this post focuses on E2E Testing for websites.</p><h2 id="the-problem">The Problem</h2><p>I was required to design a framework to perform different E2E tests on different websites. More precisely, I needed to make some tests over specific React components inside those websites. Every component has the same structure and CSS selectors no matter the website and just changes slightly from one site to another. We needed to make tests for every possible viewport (mobile, tablet, and desktop), and the components change their structure when the viewport changes.</p><p>Furthermore, I know nothing about the developers. Thus, I needed to be prepared to manage some unforeseen changes in the interface relatively easy. In other words, one critical requirement the framework needed to fulfill was to be easy to maintain.</p><p>How to make an E2E Testing Framework that doesn't care too much whether developers changed the id attribute of some button that is clicked in some test? How to be able to write tests for some component that is not created yet? And, if possible, how to make every test easy to read and understand?</p><p>All those goals can be achieved by applying some abstractions and design patterns. Here we go!</p><h2 id="the-page-object-model">The Page Object Model</h2><p>The first thing we need to do is to create an abstraction for a page. This is important for several reasons. It will increase readability. For example, you don't want to have a line in your test that reads <code>tool.getByCssSelector("button.btn.btn-submit").click()</code>, instead you want to have a line like this one: <code>page.clickSubmitLoginFormButton()</code> or something similar. You also need to keep all the CSS selectors and DOM related stuff in a single place, this way, when something in the interface is changed you only need to modify one single file (or maybe two, but not more ;-) ).</p><p>That abstraction is called the <strong>Page Object Model</strong>. You make a class that represents only the elements that you are interested in from the page. You put all the DOM related stuff in those classes.</p><p>In my case, I did it slightly different. I created two classes for every page. A <strong>PageModel</strong> and a <strong>Page Object</strong>. In the first one, I put the elements of the page. For example, suppose we are testing a login page, then my <strong>LoginPageModel</strong> would be like:</p><pre><code class="lang-pseudocode">class LoginPageModel constructor(tool) this.tool = tool loginUsernameInput() return this.tool.getById('username-input') loginPasswordInput() return this.tool.getById('password-input) loginSubmitButton() return this.tool.getById('submit-login-button')</code></pre><p>If any of those elements change in the future we only need to modify the corresponding <strong>PageModel</strong> class.</p><p>In the <strong>PageObject</strong> class, I add the actions that you can perform on the page. An example of a <strong>LoginPageObject</strong> class would be:</p><pre><code class="lang-pseudocode">class LoginPageObject constructor(pageModel) this.model = pageModel typeUsername(username) this.model.loginUsernameInput().type(username) typePassword(password) this.model.loginPasswordInput().type(password) clickLoginSubmitButton() this.model.loginSubmitButton().click()</code></pre><p>Here we can get advantage of a statically typed language that can get all the methods of the model class in compilation time, that way some IntelliSense tool can remind us the name of every method that represents a page element, and we also get more compilation errors and fewer runtime errors, which is very good for us and our mental health.</p><p>Why to separate page elements from page actions? A single class that contains both the elements and the actions can be very large. We can say that by doing this we are applying the Single Responsibility Principle and that would be cool, but in this case, that has not too many practical significance beyond readability and keeping classes simple.</p><p>With the <strong>Page Object</strong> abstraction we can make tests that only depend on page objects instead of writing some tricky CSS selectors in the middle of the test code. We keep all the DOM related stuff in a single place and our tests can be more expressive and easy to understand.</p><h2 id="tests-the-facade-pattern">Tests. The Facade Pattern</h2><p>Now we have many classes that contain all the elements and actions of several pages. What we need to do now is to build our tests. Those tests will provide a simple interface that exposes to the client the <code>run</code> functionality. This functionality returns a test result. The client doesn't have to worry about accessing any element or doing any action, just needs to instantiate the test and run it.</p><p>When we provide a simple interface that hides a more complex infrastructure we are applying the Facade Pattern. I know that's only a fancy name for something that is clear we needed to do. Continuing with our Login Page Test example, the <strong>LoginTest</strong> would be something like this:</p><pre><code class="lang-pseudocode">class LoginTest constructor(loginPageObject) this.pageObject = loginPageObject run() this.pageObject.typeUsername("TestUser") this.pageObject.typePassword("TestPassword") this.pageObject.clickLoginSubmitButton() assert that the login was successful</code></pre><p>The last line of the <code>run</code> method is an assertion. Depending on the complexity of the assertions you use, you can either define them separately or inside the Page Object. By choosing the first option you can reuse and extend assertions, but if your assertions are very specific for each case and simple enough the first option can be overkill and you probably will be good with the second one.</p><p>We are also injecting the Page Object dependency in the Test. We are not doing <code>this.pageObject = new LoginPageObject()</code> but receiving the dependency as an argument in the constructor. This is called <em>Dependency Injection</em>. That way, we can instantiate the same test for another page. We also inject the Page Model in Page Object instances. Then, we can have the same Page Object with another model (eg: same LoginPageObject instance with a LoginMobilePageModel instead of a regular LoginPageModel).</p><p>But now, to instantiate a test, we need to instantiate one or more Page Models, then one or more Page Objects, and finally the Test. It seems too much work. That's precisely one of the drawbacks of using Dependency Injection, but it is solvable!</p><h2 id="the-factory-pattern">The Factory Pattern</h2><p>If it's difficult, let's delegate the responsibility to another abstraction. In this case, we'll make some Factories. These are classes that are used to instantiate other classes. Every Factory class will be responsible for instantiating a specific test. That's the Factory Pattern in action!</p><p>So we can create a <strong>LoginTestFactory</strong> for our LoginTest:</p><pre><code class="lang-pseudocode">import toolclass LoginTestFactory create(config) if config.viewport == 'mobile' then return new LoginTest(new LoginPageObject(new LoginMobilePageModel(tool))) else return new LoginTest(new LoginPageObject(new LoginPageModel(tool)))</code></pre><p>Here we are representing with <code>tool</code> any possible technology you could use to get the elements of a page and interact with them. Maybe you don't pass the imported tool as is, but you create some objects using that tool and then pass those objects as parameters. But the idea is that all the relatively complex logic to make an instance of a Test is encapsulated in a Factory object.</p><p>To run our test we only need to do something like this:</p><pre><code class="lang-pseudocode">runLoginTestDesktop() factory = new LoginTestFactory() config = new ConfigObject(viewport = 'desktop') test = factory.create(config) test.run()runLoginTestMobile() factory = new LoginTestFactory() config = new ConfigObject(viewport = 'mobile') test = factory.create(config) test.run()</code></pre><p>Now, in the Conclusions section, we'll check whether we have accomplished our initial goals</p><h2 id="conclusions">Conclusions</h2><p>Building our framework in the way I have shown you in this post, can dramatically decrease the cost of a change in the user interface. All the code that depends on the user interface is isolated in specific classes that abstract the concept of a page.</p><p>That abstraction also allows us to write the tests for the next week! I mean the test for components that have not been created yet. We just make the required new PageModels and PageObjects to mock the elements on the page that will be created and we can build the rest of the process in the same way we have seen so far. When we had the specific elements on the interface we can change the page models and verify whether the application behaves as expected.</p><p>We also have tests that are very easy to read and understand since we make expressive actions like <code>this.pageObject.clickLoginSubmitButton()</code>. Thus, our tests can describe the requirements of our application and can be easily maintained.</p><p>E2E testing automation is difficult because is difficult to keep it simple, and a complex test is not a test. In this post, I have shown some design patterns and good practices to make it smoother. I have tried to make it language and tool agnostic in order you can apply these practices in your project no matter what language or technology you are using. I only assumed an Object-Oriented programming language.</p><p>No matter if you are not making an E2E Testing Framework, I made this post because I think some of these tricks can be applied in a relatively wide variety of problems.</p><p>Feel free to comment any suggestion, impression, or doubt about this article below. You can also follow me on Twitter for debates about related topics.</p>]]><![CDATA[Make your message stronger with better slides]]>https://blog.jayjaydev.com/make-your-message-stronger-with-better-slideshttps://blog.jayjaydev.com/make-your-message-stronger-with-better-slidesTue, 06 Oct 2020 02:18:59 GMT<![CDATA[<p>Is a widespread misconception to think that people are insterested in what are you talking about. A presentation is, first of all, a fight to get the total attention from the audience. In this post I will focus on how to improve the quality of your slides by giving you some tips. Maybe I will write something about other aspects of presentations in another post.</p><p>So, let's get started!</p><h2 id="1-your-slides-must-not-contain-all-the-info">1: Your slides must not contain all the info</h2><p>As a rule of thumb, if someone can read your slides and get all the info, then you're doing it wrong.</p><p>Your info must be split among the slides and the discourse. If one of them is missing so it is part of the info.</p><p>That's because your slides need to deliver information in a way that make your discourse lighter and easier to understand for the audience (that doesn't care about you).</p><h2 id="2-always-add-a-title">2: Always add a title</h2><p>No matter what is showed in that specific slide, you need to add a title to it.</p><p>If someone in the audience gets distracted at some point of your presentation, and try to return and understand your explanation, it would be better that he could see what are you talking about right now. That's the main purpose of the slide title.</p><p>Hence, you need your titles to be concise, and you need they to tell the audience what you are talking about right now. For example, "Prerequisites", "Problem definition", "State of the art", etc.</p><h2 id="3-minimize-text-maximize-images-and-graphs">3: Minimize text, maximize images and graphs</h2><p>If you want to explain an algorithm, do it by using animations and images. If you want to explain a process that has many stages, use illustrations for every stage. Your audience will appreciate that.</p><p>You now what they say: <em>A picture is worth a thousand words</em>. People don't want to read, they don't even want to understand you! You need to make your message important and interesting to them. And you definitely want your message to be as easy to digest as possible.</p><h2 id="4-keep-moving">4: Keep moving</h2><p>More than 10-15 secs without any change in your slide... Baaang!!! A half of the room is sleeping and the other half in their social media. Yes, your family included.</p><p>That doesn't mean you need to move from one slide to another with that frequency, just make something appear or disappear, or some animation perhaps, but keep moving. Maintain the audience engaged.</p><h2 id="5-the-7-rule">5: The 7 rule</h2><p>When including text just write up to 7 lines per slide and up to 7 words per line.</p><p>Try not to exceed those limits, but you can exceed them by one or two words/lines in some special cases.</p><p>Adding too much text is bad for many reasons. We have stated earlier that people don't want to read, we (as audience) prefer other smoother channels. But another drawback related with adding too much text is explained in the next tip.</p><h2 id="6-dont-show-all-the-text-at-once">6: Don't show all the text at once</h2><p>People don't want to read, but if you put some text in your slide they prefer to read it instead of paying attention to your discourse.</p><p>If you want people to keep listening to you, show the text with the info that you are talking about right now. After that, show the next line about the next topic, and so on.</p><p>When people in the audience read a lot of text from the slide, they get lost when returning to your speech. They read much faster than you speak. Just apply the brake.</p><h2 id="7-maintain-a-table-of-contents">7: Maintain a table of contents</h2><p><em>Just for larger presentations</em></p><p>Your ideas are arranged in some order in your head. Transmit that order relation to your audience explicitly. Before and after talking about some topic, show your Table of Contents.</p><p>Make sure to highlight the next topic you will be talking about and differentiate the topics you have explained already. Use different colors and opacities maybe.</p><h2 id="8-end-your-presentation-with-the-first-slide">8: End your presentation with the first slide</h2><p>You did it quite well! You managed to catch the attention of the audience during all the presentation. They are amazed and wondering how can they reach you. Maybe an email or some social media links? Nop, all they can see is a "Thank you", that is what your last slide says.</p><p>Always end your presentation with your very first slide, that which contains your name, the title of your presentation, and your contact info.</p><h2 id="important-note">Important note</h2><p>A presentation is not a bunch of slides. A presentation is a conception, a method you elaborate to transmit your ideas and results.</p><p>The answer to the question "Have you finished your presentation?" can be "Yes, I just have to finish my slides".</p><p>But slides can make the difference between a great and a poor presentation. When used in the right way, they can be a very effective weapon, but they also can make your presentation a total disaster if you use them badly.</p><h2 id="conclusions">Conclusions</h2><p>I have given you some tips to make your slides an effective weapon.</p><p>Remember to approach your presentation as a fight to get attention from your audience. With that in mind, try to make your message easy to digest, and add dynamism to your slides. That's the way to maintain people engaged.</p><p>Let me know if you enjoyed this post by reacting. Feel free to comment any doubts or recommendations. You can also follow me on Twitter for more content like this.</p>]]><![CDATA[<p>Is a widespread misconception to think that people are insterested in what are you talking about. A presentation is, first of all, a fight to get the total attention from the audience. In this post I will focus on how to improve the quality of your slides by giving you some tips. Maybe I will write something about other aspects of presentations in another post.</p><p>So, let's get started!</p><h2 id="1-your-slides-must-not-contain-all-the-info">1: Your slides must not contain all the info</h2><p>As a rule of thumb, if someone can read your slides and get all the info, then you're doing it wrong.</p><p>Your info must be split among the slides and the discourse. If one of them is missing so it is part of the info.</p><p>That's because your slides need to deliver information in a way that make your discourse lighter and easier to understand for the audience (that doesn't care about you).</p><h2 id="2-always-add-a-title">2: Always add a title</h2><p>No matter what is showed in that specific slide, you need to add a title to it.</p><p>If someone in the audience gets distracted at some point of your presentation, and try to return and understand your explanation, it would be better that he could see what are you talking about right now. That's the main purpose of the slide title.</p><p>Hence, you need your titles to be concise, and you need they to tell the audience what you are talking about right now. For example, "Prerequisites", "Problem definition", "State of the art", etc.</p><h2 id="3-minimize-text-maximize-images-and-graphs">3: Minimize text, maximize images and graphs</h2><p>If you want to explain an algorithm, do it by using animations and images. If you want to explain a process that has many stages, use illustrations for every stage. Your audience will appreciate that.</p><p>You now what they say: <em>A picture is worth a thousand words</em>. People don't want to read, they don't even want to understand you! You need to make your message important and interesting to them. And you definitely want your message to be as easy to digest as possible.</p><h2 id="4-keep-moving">4: Keep moving</h2><p>More than 10-15 secs without any change in your slide... Baaang!!! A half of the room is sleeping and the other half in their social media. Yes, your family included.</p><p>That doesn't mean you need to move from one slide to another with that frequency, just make something appear or disappear, or some animation perhaps, but keep moving. Maintain the audience engaged.</p><h2 id="5-the-7-rule">5: The 7 rule</h2><p>When including text just write up to 7 lines per slide and up to 7 words per line.</p><p>Try not to exceed those limits, but you can exceed them by one or two words/lines in some special cases.</p><p>Adding too much text is bad for many reasons. We have stated earlier that people don't want to read, we (as audience) prefer other smoother channels. But another drawback related with adding too much text is explained in the next tip.</p><h2 id="6-dont-show-all-the-text-at-once">6: Don't show all the text at once</h2><p>People don't want to read, but if you put some text in your slide they prefer to read it instead of paying attention to your discourse.</p><p>If you want people to keep listening to you, show the text with the info that you are talking about right now. After that, show the next line about the next topic, and so on.</p><p>When people in the audience read a lot of text from the slide, they get lost when returning to your speech. They read much faster than you speak. Just apply the brake.</p><h2 id="7-maintain-a-table-of-contents">7: Maintain a table of contents</h2><p><em>Just for larger presentations</em></p><p>Your ideas are arranged in some order in your head. Transmit that order relation to your audience explicitly. Before and after talking about some topic, show your Table of Contents.</p><p>Make sure to highlight the next topic you will be talking about and differentiate the topics you have explained already. Use different colors and opacities maybe.</p><h2 id="8-end-your-presentation-with-the-first-slide">8: End your presentation with the first slide</h2><p>You did it quite well! You managed to catch the attention of the audience during all the presentation. They are amazed and wondering how can they reach you. Maybe an email or some social media links? Nop, all they can see is a "Thank you", that is what your last slide says.</p><p>Always end your presentation with your very first slide, that which contains your name, the title of your presentation, and your contact info.</p><h2 id="important-note">Important note</h2><p>A presentation is not a bunch of slides. A presentation is a conception, a method you elaborate to transmit your ideas and results.</p><p>The answer to the question "Have you finished your presentation?" can be "Yes, I just have to finish my slides".</p><p>But slides can make the difference between a great and a poor presentation. When used in the right way, they can be a very effective weapon, but they also can make your presentation a total disaster if you use them badly.</p><h2 id="conclusions">Conclusions</h2><p>I have given you some tips to make your slides an effective weapon.</p><p>Remember to approach your presentation as a fight to get attention from your audience. With that in mind, try to make your message easy to digest, and add dynamism to your slides. That's the way to maintain people engaged.</p><p>Let me know if you enjoyed this post by reacting. Feel free to comment any doubts or recommendations. You can also follow me on Twitter for more content like this.</p>]]><![CDATA[Algorithms: when can I tell they're good enough?]]>https://blog.jayjaydev.com/algorithms-when-can-i-tell-theyre-good-enoughhttps://blog.jayjaydev.com/algorithms-when-can-i-tell-theyre-good-enoughSat, 03 Oct 2020 01:14:04 GMT<![CDATA[<p>There are lot of people designing algorithms to solve very hard problems efficiently. When we say <em>efficiently</em> we mean spending as few resources as possible, and with resources we mean time and memory (mainly). But how do we know whether some algorithm is the best one? How to know whether certain algorithm cannot be improved any further? In this post I am going to talk about how to prove an algorithm is as good as can be.</p><h2 id="requirements-and-some-notes">Requirements and some notes</h2><p>I assume you have some knowledge about big-O notation and algorithm design.</p><p>In this post I focus on <em>time complexity</em>, so I am going to give you some methods to demonstrate an algorithm is "as fast as can be". Some of that ideas are also applicable to memory complexity analysis.</p><p>When I say that an algorithm <code>A</code> such that <code>T(A) = O(f(x))</code> (i.e runtime complexity function of algorithm <code>A</code> is <code>O(f(x))</code>) cannot be improved any further, I mean that there is no an equivalent algorithm <code>B</code> such that <code>T(B) = O(g(x))</code> and <code>T(A) != O(g(x))</code> (i.e runtime complexity function of algorithm <code>A</code> is worse than <code>O(g(x))</code>).</p><p>Most of the time you'll be required to design an algorithm that run in a certain amount of time for a certain bounded input. So in most cases you don't need to find an optimum algorithm. But this post will give you some tricks you can use to decide whether you should try to fulfill the requirements or just change your requirements since it can be proved that those requirements are impossible to fulfill.</p><h2 id="first-with-no-tricks-at-all">First with no "tricks" at all</h2><p>I'm going to talk about some "tricks" in the next sections but let's talk about the straight way first. Given the problem, and some restrictions maybe, let's demonstrate we can't design an algorithm that ran (asymptotically) faster than a given one.</p><p>I'll just ilustrate one of those demonstrations. Let's demonstrate that it's not possible to sort a list doing just comparisons between the elements of the list in less than <code>O(n * log n)</code>.</p><h3 id="the-proof">The proof</h3><p>A permutation of a set of elements is an arrangement of those elements. For a set of size <code>n</code> there are <code>n!</code> different arrangements (<code>n*(n-1)*...*2*1</code>). So the task is, given a permutation, how to find the sorted permutation using just comparisons? When I say using just comparisons I mean that the only way to gain some knowledge about the input permutation is by comparing two elements of that permutation.</p><p>We'll represent such comparison-based algorithm as a binary tree, where the leaves are the different permutations and every interior node represents a comparison between two specific elements of the input. Here's an example of a tree for an input of length 3.</p><p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1601675600727/cs8HkEMb_.png" alt="Screenshot - sortLB.pdf - Google Chrome.png" /></p><p>So we start doing the comparison represented in the root, if the answer is "yes", then we know the permutation we're looking for is in the left subtree, otherwise it's in the right subtree. We continue applying the comparisons until we reach a leaf, which represents the sorted permutation.</p><blockquote><p>For example the leaf <code><2,3,1></code> represents the permutation where the first element of the input is in the third position, the second element is in the first position and the third element is in the second position.</p></blockquote><p>Then the length of the path from the root to one of the leaves is the amount of comparisons that we need to do to sort an specific input permutation. So, the longest possible path represents the worst case since by following it we need to do the bigger amount of comparisons. Let's demonstrate that the longest path will always contain at least <code>O(n * log n)</code> comparisons.</p><p>We know the number of different permutations is <code>n!</code> so that's the same number of different leaves in the tree. But if the longest path in the tree has length <code>h</code> then the number of leaves cannot exceed <code>2^h</code> (<code>2</code> to the power of <code>h</code>). Then we can apply some math like it's shown in the image.</p><p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1601676651508/9Y9PdCOtv.png" alt="Screenshot - sortLB.pdf - Google Chrome (2).png" /></p><p>And that <code>n/2 * log(n/2)</code> is <code>O(n*log n)</code>.</p><p>So here we represented the algorithm operations through an abstraction (a binary tree) and based on that we demonstrated that there is always a case in wich the algorithm has to make certain amount of operations (<code>O(n * log n)</code>), thus we'll never get an equivalent algorithm that can do less than that certain amount of operations.</p><blockquote><p>Does that mean that we can't sort in less than <code>O(n * log n)</code>? No! We assumed that the algorithm just makes comparisons to gain knowledge about the input (like <em>bubble sort</em>, <em>quick sort</em> and <em>merge sort</em> do). But there are sorting algorithms that assume some prior knowledge about the input and can sort in <code>O(n)</code>! Take a look at <a target="_blank" href="https://en.wikipedia.org/wiki/Counting_sort">counting sort</a> and <a target="_blank" href="https://en.wikipedia.org/wiki/Radix_sort">radix sort</a>.</p></blockquote><h2 id="the-adversary-trick">The adversary trick</h2><p>The first "trick" I'm going to talk about is known as the "Adversarial Lower Bound Technique". Now, there will be an adversary (or devil) <code>D</code>. The adversary has access to all the possible inputs, and we'll see the execution of the algorithm <code>A</code> as a game in which <code>D</code> is suposed to select one input, then <code>A</code> will ask for the result of one operation on the input to <code>D</code>, and <code>D</code> must answer the result. The game ends when <code>A</code> has an output given the input <code>D</code> has selected.</p><p>The trick here is that <code>D</code> can always change its mind about the input as long as at the end of the game the input selected by <code>D</code> was consistent with the answer <code>D</code> gave throughout the game. The task is to design an <em>adversary strategy</em> that forces <code>A</code> to ask at least as many times as we want.</p><h3 id="example">Example</h3><p>Let's consider the problem of finding the maximum in a list of length <code>n</code>. The algorithm <code>A</code> is allowed to ask whether the number at a certain position in the list is greater than the current maximum, assuming that the initial maximum is the number at a certain fixed position in the list. Let's prove that <code>D</code> can force <code>A</code> to make at least <code>n-1</code> operations.</p><p>The strategy of <code>D</code> would be place in every position <code>A</code> asks about, a bigger number every time. And the answer will always be "NO" (i.e the number your asking about is not greater than the current maximum). So, after <code>n-1</code> operations <code>A</code> can state that the maximum is the number in the position it has not asked about.</p><p>Let's consider the problem for a list of length <code>4</code>. The initial maximum will be the number in the first position. Then <code>A</code> asks to <code>D</code> whether the second number is greater than the current maximum. <code>D</code> answers "NO" and puts the number <code>1</code> in the second position of the list. Then <code>A</code> asks whether the third number is greater than the current maximum and <code>D</code> answers "NO" and put the number <code>2</code> in the third position of the list. Finally <code>A</code> asks about the fourth position and <code>D</code> answers again "NO" and put the number <code>3</code> in that position, but then <code>A</code> can state that the maximum is the first number since there is no other number greater than it, then <code>D</code> put a <code>4</code> in the first position and shows the input to <code>A</code>. That input (<code>4, 1, 2, 3</code>) is consistent with all the answers <code>D</code> gave to <code>A</code> so we found a case where <code>A</code> is forced to make <code>3</code> operations to find the maximum in a list of <code>4</code> elements.</p><p>The demonstration for a list of <code>n</code> elements is just a generalization of the method described above.</p><blockquote><p>This was a simple example. You can try to prove the lower bound of sorting algorithms described in the previous section via Adversary Technique for a more challenging demonstration.</p></blockquote><h2 id="problem-reduction">Problem reduction</h2><p>The last "trick" I am going to talk about is the Problem Reduction Technique. The idea in this case is to prove that if we could solve certain problem in time less than <code>T(n)</code> then we could solve another problem in a time we already know is impossible.</p><p>For example, we have the problem of the Epsilon Distance, which is about telling whether in a list of numbers there are two elements <code>x</code> and <code>y</code> such that <code>|x-y| < e</code> for a fixed <code>e</code>. This problem is known to have a runtime complexity that is <code>O(n * log n)</code> where <code>n</code> is the length of the list. We won't prove that in this article.</p><p>Instead we are going to prove that the problem of finding the closest pair of points among a set of <code>n</code> points in a plane cannot be solved with less than <code>O(n * log n)</code> operations.</p><p>Supose we can solve the Closest Pair of Points Problem in less than <code>O(n * log n)</code>. Now think about the Epsilon Distance Problem, if we have a list of <code>n</code> numbers <code>x1, x2, ..., xn</code>, then we can transform that list into <code>(x1, 0), (x2, 0), ..., (xn, 0)</code>. That transformation can be done with <code>O(n)</code> operations. Then we can solve the Closest Pair of Points Problem in less than <code>O(n * log n)</code> and if the distance between those points is less than <code>e</code> we can answer "YES", otherwise we answer "NO". And we have solved the Epsilon Distance Problem with less than <code>O(n * log n)</code> operations. But that's impossible! Since we know that <code>O(n * log n)</code> is the lower bound for solving that problem. Hence, we cannot solve the Closest Pair of Points Probem with less than <code>O(n * log n)</code> operations neither. </p><h2 id="np-completeness">NP-completeness</h2><p>Before the end of this article I want to talk about problems that are NP-complete. Making an oversimplification we'll say that an NP-complete problem is a problem for which we haven't found a solution that makes a polynomial amount of operations. So, until now, the known lower bound for the amount of operations those solutions perform is an exponential function of the input.</p><p>Maybe you have heard about NP-complete problems before, and probably you have been told that if one can find a polynomial solution for any of those problems then you'd be proving that all of them have a polynomial solution as well (and then you'd be solving one of the hardest problems in the history of humanity, and your name would be written in any single book about computer science, and you'd be millionare, among other benefits).</p><p>That's because the way we can prove that a problem is NP-complete is by proving that if we find a polynomial solution for the problem then we can solve another already known NP-complete problem in polynomial time. In other words, to prove a problem is NP-complete we use the reduction "trick" we've seen in the previous section.</p><p>Note that in this case we are not proving that a better runtime is not possible, we are just proving that the humanity hasn't found a polynomial solution for that problem so far.</p><p>Maybe you are wondering how the first NP-complete problem was discovered. Well, I can tell you when: 1971. I also can tell you who: Stephen Cook. I even can tell you what: the 3-SAT problem. But the how is up to you.</p><h2 id="conclusions">Conclusions</h2><p>Proving the lower bound in runtime complexity of an algorithm can be a very dificult task. In this post I have talked about some examples and "tricks" to do so.</p><p>I have made a lot of oversimplications all along the article. If you want to learn about this topic in a more technical and formal way you can search other good sources in internet. You can also leave your comment with any doubt or recommendation. You also can @-me on Twitter to start a fruitful debate or you can follow me for some tweets about computer science.</p>]]><![CDATA[<p>There are lot of people designing algorithms to solve very hard problems efficiently. When we say <em>efficiently</em> we mean spending as few resources as possible, and with resources we mean time and memory (mainly). But how do we know whether some algorithm is the best one? How to know whether certain algorithm cannot be improved any further? In this post I am going to talk about how to prove an algorithm is as good as can be.</p><h2 id="requirements-and-some-notes">Requirements and some notes</h2><p>I assume you have some knowledge about big-O notation and algorithm design.</p><p>In this post I focus on <em>time complexity</em>, so I am going to give you some methods to demonstrate an algorithm is "as fast as can be". Some of that ideas are also applicable to memory complexity analysis.</p><p>When I say that an algorithm <code>A</code> such that <code>T(A) = O(f(x))</code> (i.e runtime complexity function of algorithm <code>A</code> is <code>O(f(x))</code>) cannot be improved any further, I mean that there is no an equivalent algorithm <code>B</code> such that <code>T(B) = O(g(x))</code> and <code>T(A) != O(g(x))</code> (i.e runtime complexity function of algorithm <code>A</code> is worse than <code>O(g(x))</code>).</p><p>Most of the time you'll be required to design an algorithm that run in a certain amount of time for a certain bounded input. So in most cases you don't need to find an optimum algorithm. But this post will give you some tricks you can use to decide whether you should try to fulfill the requirements or just change your requirements since it can be proved that those requirements are impossible to fulfill.</p><h2 id="first-with-no-tricks-at-all">First with no "tricks" at all</h2><p>I'm going to talk about some "tricks" in the next sections but let's talk about the straight way first. Given the problem, and some restrictions maybe, let's demonstrate we can't design an algorithm that ran (asymptotically) faster than a given one.</p><p>I'll just ilustrate one of those demonstrations. Let's demonstrate that it's not possible to sort a list doing just comparisons between the elements of the list in less than <code>O(n * log n)</code>.</p><h3 id="the-proof">The proof</h3><p>A permutation of a set of elements is an arrangement of those elements. For a set of size <code>n</code> there are <code>n!</code> different arrangements (<code>n*(n-1)*...*2*1</code>). So the task is, given a permutation, how to find the sorted permutation using just comparisons? When I say using just comparisons I mean that the only way to gain some knowledge about the input permutation is by comparing two elements of that permutation.</p><p>We'll represent such comparison-based algorithm as a binary tree, where the leaves are the different permutations and every interior node represents a comparison between two specific elements of the input. Here's an example of a tree for an input of length 3.</p><p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1601675600727/cs8HkEMb_.png" alt="Screenshot - sortLB.pdf - Google Chrome.png" /></p><p>So we start doing the comparison represented in the root, if the answer is "yes", then we know the permutation we're looking for is in the left subtree, otherwise it's in the right subtree. We continue applying the comparisons until we reach a leaf, which represents the sorted permutation.</p><blockquote><p>For example the leaf <code><2,3,1></code> represents the permutation where the first element of the input is in the third position, the second element is in the first position and the third element is in the second position.</p></blockquote><p>Then the length of the path from the root to one of the leaves is the amount of comparisons that we need to do to sort an specific input permutation. So, the longest possible path represents the worst case since by following it we need to do the bigger amount of comparisons. Let's demonstrate that the longest path will always contain at least <code>O(n * log n)</code> comparisons.</p><p>We know the number of different permutations is <code>n!</code> so that's the same number of different leaves in the tree. But if the longest path in the tree has length <code>h</code> then the number of leaves cannot exceed <code>2^h</code> (<code>2</code> to the power of <code>h</code>). Then we can apply some math like it's shown in the image.</p><p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1601676651508/9Y9PdCOtv.png" alt="Screenshot - sortLB.pdf - Google Chrome (2).png" /></p><p>And that <code>n/2 * log(n/2)</code> is <code>O(n*log n)</code>.</p><p>So here we represented the algorithm operations through an abstraction (a binary tree) and based on that we demonstrated that there is always a case in wich the algorithm has to make certain amount of operations (<code>O(n * log n)</code>), thus we'll never get an equivalent algorithm that can do less than that certain amount of operations.</p><blockquote><p>Does that mean that we can't sort in less than <code>O(n * log n)</code>? No! We assumed that the algorithm just makes comparisons to gain knowledge about the input (like <em>bubble sort</em>, <em>quick sort</em> and <em>merge sort</em> do). But there are sorting algorithms that assume some prior knowledge about the input and can sort in <code>O(n)</code>! Take a look at <a target="_blank" href="https://en.wikipedia.org/wiki/Counting_sort">counting sort</a> and <a target="_blank" href="https://en.wikipedia.org/wiki/Radix_sort">radix sort</a>.</p></blockquote><h2 id="the-adversary-trick">The adversary trick</h2><p>The first "trick" I'm going to talk about is known as the "Adversarial Lower Bound Technique". Now, there will be an adversary (or devil) <code>D</code>. The adversary has access to all the possible inputs, and we'll see the execution of the algorithm <code>A</code> as a game in which <code>D</code> is suposed to select one input, then <code>A</code> will ask for the result of one operation on the input to <code>D</code>, and <code>D</code> must answer the result. The game ends when <code>A</code> has an output given the input <code>D</code> has selected.</p><p>The trick here is that <code>D</code> can always change its mind about the input as long as at the end of the game the input selected by <code>D</code> was consistent with the answer <code>D</code> gave throughout the game. The task is to design an <em>adversary strategy</em> that forces <code>A</code> to ask at least as many times as we want.</p><h3 id="example">Example</h3><p>Let's consider the problem of finding the maximum in a list of length <code>n</code>. The algorithm <code>A</code> is allowed to ask whether the number at a certain position in the list is greater than the current maximum, assuming that the initial maximum is the number at a certain fixed position in the list. Let's prove that <code>D</code> can force <code>A</code> to make at least <code>n-1</code> operations.</p><p>The strategy of <code>D</code> would be place in every position <code>A</code> asks about, a bigger number every time. And the answer will always be "NO" (i.e the number your asking about is not greater than the current maximum). So, after <code>n-1</code> operations <code>A</code> can state that the maximum is the number in the position it has not asked about.</p><p>Let's consider the problem for a list of length <code>4</code>. The initial maximum will be the number in the first position. Then <code>A</code> asks to <code>D</code> whether the second number is greater than the current maximum. <code>D</code> answers "NO" and puts the number <code>1</code> in the second position of the list. Then <code>A</code> asks whether the third number is greater than the current maximum and <code>D</code> answers "NO" and put the number <code>2</code> in the third position of the list. Finally <code>A</code> asks about the fourth position and <code>D</code> answers again "NO" and put the number <code>3</code> in that position, but then <code>A</code> can state that the maximum is the first number since there is no other number greater than it, then <code>D</code> put a <code>4</code> in the first position and shows the input to <code>A</code>. That input (<code>4, 1, 2, 3</code>) is consistent with all the answers <code>D</code> gave to <code>A</code> so we found a case where <code>A</code> is forced to make <code>3</code> operations to find the maximum in a list of <code>4</code> elements.</p><p>The demonstration for a list of <code>n</code> elements is just a generalization of the method described above.</p><blockquote><p>This was a simple example. You can try to prove the lower bound of sorting algorithms described in the previous section via Adversary Technique for a more challenging demonstration.</p></blockquote><h2 id="problem-reduction">Problem reduction</h2><p>The last "trick" I am going to talk about is the Problem Reduction Technique. The idea in this case is to prove that if we could solve certain problem in time less than <code>T(n)</code> then we could solve another problem in a time we already know is impossible.</p><p>For example, we have the problem of the Epsilon Distance, which is about telling whether in a list of numbers there are two elements <code>x</code> and <code>y</code> such that <code>|x-y| < e</code> for a fixed <code>e</code>. This problem is known to have a runtime complexity that is <code>O(n * log n)</code> where <code>n</code> is the length of the list. We won't prove that in this article.</p><p>Instead we are going to prove that the problem of finding the closest pair of points among a set of <code>n</code> points in a plane cannot be solved with less than <code>O(n * log n)</code> operations.</p><p>Supose we can solve the Closest Pair of Points Problem in less than <code>O(n * log n)</code>. Now think about the Epsilon Distance Problem, if we have a list of <code>n</code> numbers <code>x1, x2, ..., xn</code>, then we can transform that list into <code>(x1, 0), (x2, 0), ..., (xn, 0)</code>. That transformation can be done with <code>O(n)</code> operations. Then we can solve the Closest Pair of Points Problem in less than <code>O(n * log n)</code> and if the distance between those points is less than <code>e</code> we can answer "YES", otherwise we answer "NO". And we have solved the Epsilon Distance Problem with less than <code>O(n * log n)</code> operations. But that's impossible! Since we know that <code>O(n * log n)</code> is the lower bound for solving that problem. Hence, we cannot solve the Closest Pair of Points Probem with less than <code>O(n * log n)</code> operations neither. </p><h2 id="np-completeness">NP-completeness</h2><p>Before the end of this article I want to talk about problems that are NP-complete. Making an oversimplification we'll say that an NP-complete problem is a problem for which we haven't found a solution that makes a polynomial amount of operations. So, until now, the known lower bound for the amount of operations those solutions perform is an exponential function of the input.</p><p>Maybe you have heard about NP-complete problems before, and probably you have been told that if one can find a polynomial solution for any of those problems then you'd be proving that all of them have a polynomial solution as well (and then you'd be solving one of the hardest problems in the history of humanity, and your name would be written in any single book about computer science, and you'd be millionare, among other benefits).</p><p>That's because the way we can prove that a problem is NP-complete is by proving that if we find a polynomial solution for the problem then we can solve another already known NP-complete problem in polynomial time. In other words, to prove a problem is NP-complete we use the reduction "trick" we've seen in the previous section.</p><p>Note that in this case we are not proving that a better runtime is not possible, we are just proving that the humanity hasn't found a polynomial solution for that problem so far.</p><p>Maybe you are wondering how the first NP-complete problem was discovered. Well, I can tell you when: 1971. I also can tell you who: Stephen Cook. I even can tell you what: the 3-SAT problem. But the how is up to you.</p><h2 id="conclusions">Conclusions</h2><p>Proving the lower bound in runtime complexity of an algorithm can be a very dificult task. In this post I have talked about some examples and "tricks" to do so.</p><p>I have made a lot of oversimplications all along the article. If you want to learn about this topic in a more technical and formal way you can search other good sources in internet. You can also leave your comment with any doubt or recommendation. You also can @-me on Twitter to start a fruitful debate or you can follow me for some tweets about computer science.</p>]]><![CDATA[My first year as a developer]]>https://blog.jayjaydev.com/my-first-year-as-a-developerhttps://blog.jayjaydev.com/my-first-year-as-a-developerWed, 30 Sep 2020 17:19:57 GMT<![CDATA[<p>It has been a year since I started to make money from building software. I don't call myself a developer, maybe one year making a living from programming is very similar to your definition of what a developer is, but I don't see the things in that way. However this could be the subject of another post. Here I will write about some of my first year experiences.</p><blockquote><p>I'll include some side notes like this one. I'll use these notes to briefly introduce some concepts and tools for those who are starting their programming journey right now.</p></blockquote><h2 id="rigidness-is-not-that-bad">Rigidness is not that bad</h2><p>When it comes to choosing a framework or a library there is often a lot of discussion. At some point in my evolution as a programmer I thought that flexibility is always good. After a bunch of lines of code I learned that rigidness is not that bad.</p><p>Frameworks, libraries, and tools in general are here because we are bad programmers by nature (among other causes). I really appreciate when a tool forces me to make things in a unique specific and very good way, so I can acomplish my goals by following that way.</p><p>In a smooth scenario, when definitions and requirements don't change abruptly during a week I always choose rigidness. I always choose the straight path.</p><h2 id="when-premises-dont-hold">When premises don't hold</h2><p>Then here I am, a guy that prefers the straight path in the middle of this jungle with no path at all. Definitions and requirements are changing every day in the most abrupt possible way and every week I am doing a very different project. I had to learn to embrace flexibility, in the hard way.</p><p>Now I am going to ilustrate that hard way through examples.</p><p>They decided to build a website using React. I learned React while doing the project so I started to consume a lot of tutorials. I decided to use <code>create-react-app</code> (CRA) to scafold the project since it is used in every single tutorial. So far, so good. I had a good workplace and was able to make the production build with a single command and so on.</p><blockquote><p>I highly recommend you to learn something while doing a project and, if possible, while getting paid for it. I think it is the best way to learn, at least when it comes to frameworks and tools in general.</p><p>CRA is a very useful way to get started with your React project. With just typing <code>npx create-react-app <project-name></code> you get the whole project scafolding. I think every React developer can take advantage from this tool no matter if you are novice or senior. I won't complain about CRA, but I think some tutorials should warn us about the possible drawbacks resulting from using it.</p></blockquote><p>Then the boss learned what a Progressive Web App (PWA) is, and after a couple of weeks we were required to make the site progressive. This is not a big deal since CRA provides us with a service worker.</p><blockquote><p>PWA is a mechanism to make websites working offline and to behave like native apps in mobile devices, among other benefits. This is achieved with the colaboration of several components like the browser, a special script called service worker, a manifest file, and some code to install the service worker in the browser. As I said, CRA provides us with the required components in order we can make our React site progressive.</p></blockquote><p>Once you have a service worker in your site, you are able to receive push notifications from a server, and of course, after two weeks we were required to implement this too. But to make that possible you need to make some adjustments in the service worker and guess what, the CRA service worker is not customizable at all!</p><blockquote><p>Maybe you have encountered some sites that ask you whether you'd like to receive notifications. If you answer yes, they can send you the so called push notifications with the content they like. This notifications pop up in your device no matter that you have the browser closed. This is possible because the browser keeps the service worker script running in the background.</p></blockquote><p>Then I had to build a new service worker from scratch and the code to install it. That is not a big deal because I had have to do it no matter I had used CRA or not. So let's move to other issues.</p><p>After building the whole payment process, they decided that we needed to use some service that verifies your identity from some images of your face and of an identity document. So we needed to change a lot of things in the already built payment process. I made possible that the user could take a selfie, and take the picture of his ID document. But after that, we were required to use a specific SDK provided by the service in order to make the images to fulfill some features required for a reliable evaluation.</p><p>The SDK was a vanilla Javascript file that needed to be imported via HTML script tag. It is intended to be integrated in applications that use vanilla javascript. You know, the non-NodeJS one. So I needed to make the impossible to get everything working without making <code>eject</code>. After knowing about webpack, I'd like to go back in time and start my project from scratch, without using CRA or anything and taking care of my own webpack configuration. Belive me, this is just a single example because I don't want to make the story so long, I just want to make my point.</p><blockquote><p>Webpack is a tool that allows us to translate our NodeJS code into a vanilla javascript bundle that can be used in our production site. It's hard to master and that's why tools like CRA manage all the webpack related issues for you. But I haven't found a smooth enough way to make changes in the CRA webpack configurations, so I think that if you are facing the sort of complexities I am telling you about, you should consider using some more flexible tool (and then tell me what tool is that) or doing your own webpack configurations. CRA allows you to run <code>npm eject</code> and after that, you are by your own with webpack and everything, but I am not brave enough yet.</p></blockquote><p>My first task in this company, before the React website I have written about above, was to build a responsive website. After a lot of effort and passing through all sort of dificulties, the site have never been used. Yes, I got paid, but this post is not about making money, this post is about what I think is wrong.</p><h2 id="what-i-would-do">What I would do</h2><p>The problem here is not the changing. There are always unstable scenarios, businesses that are unstable by nature, or at least that become unstable due to new events. The problem is the rush in the decision making, the lack of connection between developers and the business men and the misconception by some of those business men that the cost of developing is negligible.</p><p>I think you just learn about good programming practices when you find yourself repeating the same code everywhere and fixing the same bug in diferent places. Well, with this experience I have learned the necessity of planning and writing. You need to define your problem and to write down that definition. You need to keep a record of your requirements, and even keep a record of the infrastructure you need in order to fulfill those requirements. I don't like to overthink, but it's just a lack of common sense thinking that you can get something by just hitting the keys when the project is complex enough.</p><p>The problem is not the need to make big changes in the project per se. But if you can include those changes in the initial plan then they are not changes but initial requirements, and the developing process is faster, cleaner and no one do fruitless job. The sooner the change is predicted, the smaller and the easier the refactoring.</p><h2 id="conclusions">Conclusions</h2><p>Those have been some of my experiences in this firs year as a developer. I'll certainly write about some others, don't think everything have been that painful.</p><p>I hope you or your boss don't be suffering the WIPCA syndrome (Why Isn't Paul Coding Anything?!). So always try to write your problem definition, the requirements and everything you think is necessary, like the infrastructure. No matter if you are using Scrum, Waterfall or whatever. Just write a little bit and try to make changes, which will always be needed, less painful. Of course this only applies to complex enough projects, but don't underestimate projects, actually a way to assert whether a project is complex enough is defining the problem and the requirements.</p><p>If you liked this post hit the like button. Feel free to comment whatever you want. You can also follow me on Twitter for debates about Computer Science.</p>]]><![CDATA[<p>It has been a year since I started to make money from building software. I don't call myself a developer, maybe one year making a living from programming is very similar to your definition of what a developer is, but I don't see the things in that way. However this could be the subject of another post. Here I will write about some of my first year experiences.</p><blockquote><p>I'll include some side notes like this one. I'll use these notes to briefly introduce some concepts and tools for those who are starting their programming journey right now.</p></blockquote><h2 id="rigidness-is-not-that-bad">Rigidness is not that bad</h2><p>When it comes to choosing a framework or a library there is often a lot of discussion. At some point in my evolution as a programmer I thought that flexibility is always good. After a bunch of lines of code I learned that rigidness is not that bad.</p><p>Frameworks, libraries, and tools in general are here because we are bad programmers by nature (among other causes). I really appreciate when a tool forces me to make things in a unique specific and very good way, so I can acomplish my goals by following that way.</p><p>In a smooth scenario, when definitions and requirements don't change abruptly during a week I always choose rigidness. I always choose the straight path.</p><h2 id="when-premises-dont-hold">When premises don't hold</h2><p>Then here I am, a guy that prefers the straight path in the middle of this jungle with no path at all. Definitions and requirements are changing every day in the most abrupt possible way and every week I am doing a very different project. I had to learn to embrace flexibility, in the hard way.</p><p>Now I am going to ilustrate that hard way through examples.</p><p>They decided to build a website using React. I learned React while doing the project so I started to consume a lot of tutorials. I decided to use <code>create-react-app</code> (CRA) to scafold the project since it is used in every single tutorial. So far, so good. I had a good workplace and was able to make the production build with a single command and so on.</p><blockquote><p>I highly recommend you to learn something while doing a project and, if possible, while getting paid for it. I think it is the best way to learn, at least when it comes to frameworks and tools in general.</p><p>CRA is a very useful way to get started with your React project. With just typing <code>npx create-react-app <project-name></code> you get the whole project scafolding. I think every React developer can take advantage from this tool no matter if you are novice or senior. I won't complain about CRA, but I think some tutorials should warn us about the possible drawbacks resulting from using it.</p></blockquote><p>Then the boss learned what a Progressive Web App (PWA) is, and after a couple of weeks we were required to make the site progressive. This is not a big deal since CRA provides us with a service worker.</p><blockquote><p>PWA is a mechanism to make websites working offline and to behave like native apps in mobile devices, among other benefits. This is achieved with the colaboration of several components like the browser, a special script called service worker, a manifest file, and some code to install the service worker in the browser. As I said, CRA provides us with the required components in order we can make our React site progressive.</p></blockquote><p>Once you have a service worker in your site, you are able to receive push notifications from a server, and of course, after two weeks we were required to implement this too. But to make that possible you need to make some adjustments in the service worker and guess what, the CRA service worker is not customizable at all!</p><blockquote><p>Maybe you have encountered some sites that ask you whether you'd like to receive notifications. If you answer yes, they can send you the so called push notifications with the content they like. This notifications pop up in your device no matter that you have the browser closed. This is possible because the browser keeps the service worker script running in the background.</p></blockquote><p>Then I had to build a new service worker from scratch and the code to install it. That is not a big deal because I had have to do it no matter I had used CRA or not. So let's move to other issues.</p><p>After building the whole payment process, they decided that we needed to use some service that verifies your identity from some images of your face and of an identity document. So we needed to change a lot of things in the already built payment process. I made possible that the user could take a selfie, and take the picture of his ID document. But after that, we were required to use a specific SDK provided by the service in order to make the images to fulfill some features required for a reliable evaluation.</p><p>The SDK was a vanilla Javascript file that needed to be imported via HTML script tag. It is intended to be integrated in applications that use vanilla javascript. You know, the non-NodeJS one. So I needed to make the impossible to get everything working without making <code>eject</code>. After knowing about webpack, I'd like to go back in time and start my project from scratch, without using CRA or anything and taking care of my own webpack configuration. Belive me, this is just a single example because I don't want to make the story so long, I just want to make my point.</p><blockquote><p>Webpack is a tool that allows us to translate our NodeJS code into a vanilla javascript bundle that can be used in our production site. It's hard to master and that's why tools like CRA manage all the webpack related issues for you. But I haven't found a smooth enough way to make changes in the CRA webpack configurations, so I think that if you are facing the sort of complexities I am telling you about, you should consider using some more flexible tool (and then tell me what tool is that) or doing your own webpack configurations. CRA allows you to run <code>npm eject</code> and after that, you are by your own with webpack and everything, but I am not brave enough yet.</p></blockquote><p>My first task in this company, before the React website I have written about above, was to build a responsive website. After a lot of effort and passing through all sort of dificulties, the site have never been used. Yes, I got paid, but this post is not about making money, this post is about what I think is wrong.</p><h2 id="what-i-would-do">What I would do</h2><p>The problem here is not the changing. There are always unstable scenarios, businesses that are unstable by nature, or at least that become unstable due to new events. The problem is the rush in the decision making, the lack of connection between developers and the business men and the misconception by some of those business men that the cost of developing is negligible.</p><p>I think you just learn about good programming practices when you find yourself repeating the same code everywhere and fixing the same bug in diferent places. Well, with this experience I have learned the necessity of planning and writing. You need to define your problem and to write down that definition. You need to keep a record of your requirements, and even keep a record of the infrastructure you need in order to fulfill those requirements. I don't like to overthink, but it's just a lack of common sense thinking that you can get something by just hitting the keys when the project is complex enough.</p><p>The problem is not the need to make big changes in the project per se. But if you can include those changes in the initial plan then they are not changes but initial requirements, and the developing process is faster, cleaner and no one do fruitless job. The sooner the change is predicted, the smaller and the easier the refactoring.</p><h2 id="conclusions">Conclusions</h2><p>Those have been some of my experiences in this firs year as a developer. I'll certainly write about some others, don't think everything have been that painful.</p><p>I hope you or your boss don't be suffering the WIPCA syndrome (Why Isn't Paul Coding Anything?!). So always try to write your problem definition, the requirements and everything you think is necessary, like the infrastructure. No matter if you are using Scrum, Waterfall or whatever. Just write a little bit and try to make changes, which will always be needed, less painful. Of course this only applies to complex enough projects, but don't underestimate projects, actually a way to assert whether a project is complex enough is defining the problem and the requirements.</p><p>If you liked this post hit the like button. Feel free to comment whatever you want. You can also follow me on Twitter for debates about Computer Science.</p>]]><![CDATA[Combining good ideas to get the best solution]]>https://blog.jayjaydev.com/combining-good-ideas-to-get-the-best-solutionhttps://blog.jayjaydev.com/combining-good-ideas-to-get-the-best-solutionSun, 27 Sep 2020 13:41:20 GMT<![CDATA[<p>Trade-offs are present in all our activities. Maybe you have heard the so-called "No free lunch theorem" in Machine Learning. The theorem states that there is no silver bullet when it comes to machine learning models, but in fact, there is never a silver bullet when it comes to anything. And that is not a computer science principle but an economics principle.</p><p>If you have written code for more than one month is almost sure you have experienced the necessity of trade-offs when programming or at least have heard about it. Sometimes you sacrifice performance in the name of security, security in the name of scalability, beauty and readability in the name of performance, and so on. Don't forget you also sacrifice parties and fun in general in the name of programming so make it worth it.</p><p>In the specific case of algorithms, the main resources are time and memory, so the trade-offs always involve those resources. It's common to find several solutions for the same problem because one of them is faster, but the other one is cheaper when it comes to storage. Of course, there are other factors like implementation, simplicity, and security. In this post, I'm going to write about combining several solutions to get the one that fulfills our requirements.</p><p>First I'm going to show an interesting idea that E.W. Dijkstra proposed to find prime numbers. After that, I will show an idea I came up with to obtain a data structure that combines the power of arrays and linked lists. </p><h2 id="requirements">Requirements</h2><p>I assume you have basic programming skills as well as some knowledge about data structures like arrays, linked lists, and heaps.</p><p>It is necessary to have some notions of big-O notation to calculate complexity.</p><p>It would be good you were familiarized with the algorithm called Sieve of Eratosthenes, in case you don't, you can check this <a target="_blank" href="https://www.geeksforgeeks.org/sieve-of-eratosthenes/">link</a>.</p><p>There is no other previous knowledge required to understand what I'm about to say.</p><h2 id="two-algorithms-one-problem">Two algorithms, one problem</h2><p>The problem is to find all prime numbers from 0 (zero) to N. The kind of problems you learn to do when you are beginning your journey in programming. And I want to start with the simplest solution.</p><p>In the naive algorithm, we iterate over all numbers <code>x</code> from 2 to N. Then we check whether <code>x</code> has any divisor besides itself and one. For the last step, we can check for every number <code>d</code> between 2 and <code>x - 1</code> whether it is a divisor or not.</p><p>There is room for improvement in the last step because we only need to check the divisors that are less than or equal to the square root of x. A pseudocode of the algorithm is written below</p><pre><code> <span class="hljs-string">primes(N):</span> <span class="hljs-string">prime_numbers</span> <span class="hljs-string"><-</span> [] <span class="hljs-comment"># empty list</span> <span class="hljs-string">for</span> <span class="hljs-string">x</span> <span class="hljs-string">=</span> <span class="hljs-attr">2 to N:</span> <span class="hljs-string">is_prime</span> <span class="hljs-string"><-</span> <span class="hljs-literal">true</span> <span class="hljs-string">for</span> <span class="hljs-string">d</span> <span class="hljs-string">=</span> <span class="hljs-number">2</span> <span class="hljs-string">to</span> <span class="hljs-string">sqrt(x):</span> <span class="hljs-attr">if d divides x:</span> <span class="hljs-string">is_prime</span> <span class="hljs-string"><-</span> <span class="hljs-literal">false</span> <span class="hljs-comment"># we found a divisor of x so x is not a prime number</span> <span class="hljs-attr">if is_prime:</span> <span class="hljs-string">prime_numbers.add(x)</span> <span class="hljs-comment"># if we didn't find a divisor then x is prime</span> <span class="hljs-string">return</span> <span class="hljs-string">prime_numbers</span></code></pre><p>What's the runtime complexity of the previous algorithm? Well, we take every number from 2 to N and for each one, we iterate over all its possible divisors, so we make <code>O(N*sqrt(N))</code> operations, where <code>sqrt</code> stands for the square root function.</p><p>What about memory? We only store the primes we find. For a big enough N the amount of primes is relatively small compared with N. So, let's denote the memory complexity as <code>O(Primes(N))</code>, where <code>Primes(N)</code> is the number of primes between 0 and N. Note that this is optimum in terms of memory because we need at least to store all the prime numbers.</p><p>Probably you already know that there is a better algorithm to find all the prime numbers in a given range: The sieve of Eratosthenes. Let's look at it.</p><p>The idea is to maintain an array of length N where every entry is either false or true. If the i-th position in the array is true, then we say that <code>i</code> is a prime number, otherwise <code>i</code> is not prime.</p><p>We start with all positions with a true value and then, for every position, starting with <code>i = 2</code> we make false every multiple of <code>i</code>. We end up with an array in which every position with a true value represents a prime number. The code with some optimizations is presented below.</p><pre><code> primes(N): prime_numbers <- [] # empty list sieve <- a <span class="hljs-type">boolean</span> list <span class="hljs-keyword">with</span> length equal <span class="hljs-keyword">to</span> N <span class="hljs-keyword">and</span> <span class="hljs-keyword">all</span> its elements <span class="hljs-keyword">set</span> <span class="hljs-keyword">to</span> <span class="hljs-keyword">true</span> <span class="hljs-keyword">for</span> i = <span class="hljs-number">2</span> <span class="hljs-keyword">to</span> sqrt(N): # we <span class="hljs-keyword">only</span> need <span class="hljs-keyword">to</span> <span class="hljs-keyword">analyze</span> the numbers less than <span class="hljs-keyword">or</span> equal <span class="hljs-keyword">to</span> sqrt(N) <span class="hljs-keyword">if</span> sieve[i]: # <span class="hljs-keyword">if</span> sieve[i] <span class="hljs-keyword">is</span> <span class="hljs-keyword">true</span> <span class="hljs-keyword">then</span> i <span class="hljs-keyword">is</span> prime prime_numbers.<span class="hljs-keyword">add</span>(i) <span class="hljs-keyword">for</span> j = i*i <span class="hljs-keyword">to</span> N: # we can <span class="hljs-keyword">start</span> the <span class="hljs-keyword">inner</span> <span class="hljs-keyword">loop</span> <span class="hljs-keyword">in</span> i*i sieve[j] <- <span class="hljs-keyword">false</span> j <- j + i <span class="hljs-keyword">return</span> prime_numbers</code></pre><p>It can be proved that the algorithm described above does <a target="_blank" href="https://www.geeksforgeeks.org/how-is-the-time-complexity-of-sieve-of-eratosthenes-is-nloglogn/">O(N log(N))</a> operations which is better than the naive approach. Even it can be improved to <a target="_blank" href="https://www.geeksforgeeks.org/sieve-eratosthenes-0n-time-complexity">O(N)</a>! But we need to store an array with all the N numbers, and thus, we end up with a more expensive algorithm in memory terms. Here we have the trade-off: time in exchange for memory.</p><p>Can we design an algorithm that is faster than the naive idea but cheaper in memory terms than the sieve of Eratosthenes? </p><h2 id="dijkstra-and-the-production-line">Dijkstra and the production line</h2><p>In the '60s, E.W. Dijkstra wrote in one of his manuscripts [<a target="_blank" href="https://www.cs.utexas.edu/~EWD/ewd02xx/EWD249.PDF">PDF</a>] an algorithm that combines the naive and the sieve ideas. But before we talk about it, let's analyze the differences between the two previous algorithms. </p><p>When applying the naive algorithm we focus on analyzing whether every number is prime or not. When applying the sieve we analyze, for every prime number, all its multiples. The difference can be illustrated with the following analogy.</p><p>Imagine we want to build several products, like action figures. We have two options: building them one by one or applying a production line and in every stage, we add one component of the action figure. With the latter, we end up producing more in less time, but we need the "line infrastructure".</p><p>Dijkstra combined those ideas by analyzing a number at a time but taking advantage of the previous operations. We can maintain a pool of primes that have been already discovered, and, for each of those prime numbers store the least multiple that has not been analyzed yet. For example:</p><p>If we are analyzing number 6, then we have stored primes 2, 3, and 5, along with their least-not-analyzed-multiples that are: 6 for 2, also 6 for 3, and 10 for 5.</p><p>Then, when analyzing a new number we take the smallest multiple stored until that moment and if that multiple is greater than the new number, then we have found a new prime, otherwise, we have a composite number and we need to update the multiples of the stored primes that have the new number as its least multiple.</p><p>We begin storing the prime number <code>2</code> with the least multiple <code>4</code> as well. Then when analyzing <code>3</code> we find that <code>4 > 3</code> so <code>3</code> is prime. We store <code>3</code> along with its smallest multiple that has not been analyzed yet (<code>6</code>). When analyzing <code>4</code> we find that <code>4</code> is stored as a multiple of <code>2</code>, then we update the multiple of <code>2</code> that now will be <code>6</code>. When analyzing <code>5</code> we find that <code>6 > 5</code> so <code>5</code> is a prime number and we store it along with <code>10</code>, and so on... The code is presented below.</p><pre><code> <span class="hljs-string">primes(N):</span> <span class="hljs-string">primes_pool</span> <span class="hljs-string"><-</span> <span class="hljs-string">Heap(</span> <span class="hljs-string">(4,</span> <span class="hljs-number">2</span><span class="hljs-string">)</span> <span class="hljs-string">)</span> <span class="hljs-comment"># a heap that contains a tuple with a prime number (2) and its least multiple that hasn't been analyzed yet (4).</span> <span class="hljs-string">prime_numbers</span> <span class="hljs-string"><-</span> [<span class="hljs-number">2</span>] <span class="hljs-comment"># a list that contains the number 2</span> <span class="hljs-string">for</span> <span class="hljs-string">i</span> <span class="hljs-string">=</span> <span class="hljs-attr">3 to N:</span> <span class="hljs-string">tuple</span> <span class="hljs-string"><-</span> <span class="hljs-string">getMin(primes_pool)</span> <span class="hljs-comment"># get the tuple with the minimum multiple, the prime number attached to it is irrelevant</span> <span class="hljs-string">mult</span> <span class="hljs-string"><-</span> <span class="hljs-string">tuple.first</span> <span class="hljs-comment"># the multiple is stored in the first position of the tuple</span> <span class="hljs-string">if</span> <span class="hljs-string">mult</span> <span class="hljs-string">></span> <span class="hljs-attr">i:</span> <span class="hljs-string">prime_numbers.add(i)</span> <span class="hljs-comment"># i is prime!</span> <span class="hljs-string">primes_pool.insert(</span> <span class="hljs-string">(i*i,</span> <span class="hljs-string">i)</span> <span class="hljs-string">)</span> <span class="hljs-comment"># we insert i along with their least multiple, but it can be inserted i*i instead and the algorithm remains correct</span> <span class="hljs-string">continue</span> <span class="hljs-comment"># otherwise i isn't prime and then...</span> <span class="hljs-attr">for t such that t is in primes_pool and t.first is equal to mult:</span> <span class="hljs-string">t.first</span> <span class="hljs-string"><-</span> <span class="hljs-string">t.first</span> <span class="hljs-string">+</span> <span class="hljs-string">t.second</span> <span class="hljs-comment"># we update every tuple in the pool that has i as it's least multiple</span> <span class="hljs-string">return</span> <span class="hljs-string">prime_numbers</span></code></pre><p>The previous method only stores the prime numbers and another number for each of those primes. So we have a memory complexity of <code>O(Primes(N))</code> which is the same as the naive idea. If we store the prime numbers along with their multiples in a structure like a heap, we get a time complexity of <code>O(N*log N)</code> which is the same as the sieve! So we got what we wanted!</p><p>The trick here is that we don't need to mark every multiple of the given prime but just the least multiple.</p><p>I need to say that this is not a practical idea in the sense that the memory complexity of the sieve of Eratosthenes is not that bad and it is an algorithm very easy to implement. My point is that sometimes you have several ideas, each of them cannot be applied because of some flaw, then maybe is a good idea to combine their strengths and you can get a hybrid applicable solution for your problem. The Factory of Primes of Dijkstra has taught me to think in that way, although I have never implemented that algorithm in real-life scenarios. </p><h2 id="combining-arrays-with-linked-lists">Combining arrays with linked lists</h2><p>Arrays are simple structures that allow us to get an element by its index in constant time. But we need <code>O(N)</code> operations to insert or remove an element to/from the array in the worst case, where N is the length of the array.</p><p>On the other hand, linked lists are structures composed of nodes. Every node has a reference to the next one, and, in the case of doubly-linked lists, each node has also a reference to the previous one. In this case, we need <code>O(N)</code> operations to reach a node in the worst case, but the insert and remove operations can be done in constant time. I think it's natural to think about a "perfect data structure" that allows us to make the three operations in constant time. Sadly, such a structure does not exist, but I have found a middle point between the two opposite poles.</p><p>The "issue" with arrays is that they maintain a reference to all of their elements. That allows us to retrieve any element with the same amount of operations no matter where the element is. But maintaining those references is what makes the insertions and deletions so costly. When it comes to linked lists we only maintain a reference to the first and the last node and each node has a reference to its neighbors. So, when inserting or removing a new element we only need to change a few references. But that lack of references is the cause that we have to spend so many operations to get an element in the worst case.</p><p>Viewing the problem from that angle, the idea of finding a middle point in the number of references seems natural. What happens if we maintain a reference to <code>sqrt(N)</code> nodes of the linked list instead of referencing just the first and the last element?</p><p>That allows us to have a list of length <code>sqrt(N)</code> such that the distance between each of those nodes in the actual list is <code>sqrt(N)</code>. Having that, we can do each operation (index, insertion, and deletion) in <code>O(sqrt(N))</code>.</p><p>If you want to see more details about this structure and a Lisp implementation of it, you can do it <a target="_blank" href="https://github.com/josejorgers/indexed-linked-list">here</a>.</p><blockquote><p><strong>Update</strong>: I have written a post about this data structure. You can read it <a target="_blank" href="https://jj.hashnode.dev/improving-linked-lists">here</a></p></blockquote><h2 id="conclusions">Conclusions</h2><p>We have seen two examples of combining existing solutions to get another one that has some of the good qualities of each of the previous ones. My purpose was to show you this way of thinking, not just specific examples. </p><p>Note that in the case of Dijkstra's idea, we could achieve the time of the fastest solution and the memory complexity of the naive algorithm. In the second example, we just got a middle point, so the fastest solution is still faster and the memory-cheaper solution is still cheaper, but the new structure is like a decathlon athlete, it is good in every specificity, but not the best in anyone. So don't try to find the silver bullet, remember the no free lunch. Even Dijkstra's idea has the disadvantage of being harder to implement and to understand.</p><p>Hope you have learned or found attractive this post. Feel free to ask whatever you want or give me some feedback, I will appreciate that a lot. Stay tuned for the next post!</p>]]><![CDATA[<p>Trade-offs are present in all our activities. Maybe you have heard the so-called "No free lunch theorem" in Machine Learning. The theorem states that there is no silver bullet when it comes to machine learning models, but in fact, there is never a silver bullet when it comes to anything. And that is not a computer science principle but an economics principle.</p><p>If you have written code for more than one month is almost sure you have experienced the necessity of trade-offs when programming or at least have heard about it. Sometimes you sacrifice performance in the name of security, security in the name of scalability, beauty and readability in the name of performance, and so on. Don't forget you also sacrifice parties and fun in general in the name of programming so make it worth it.</p><p>In the specific case of algorithms, the main resources are time and memory, so the trade-offs always involve those resources. It's common to find several solutions for the same problem because one of them is faster, but the other one is cheaper when it comes to storage. Of course, there are other factors like implementation, simplicity, and security. In this post, I'm going to write about combining several solutions to get the one that fulfills our requirements.</p><p>First I'm going to show an interesting idea that E.W. Dijkstra proposed to find prime numbers. After that, I will show an idea I came up with to obtain a data structure that combines the power of arrays and linked lists. </p><h2 id="requirements">Requirements</h2><p>I assume you have basic programming skills as well as some knowledge about data structures like arrays, linked lists, and heaps.</p><p>It is necessary to have some notions of big-O notation to calculate complexity.</p><p>It would be good you were familiarized with the algorithm called Sieve of Eratosthenes, in case you don't, you can check this <a target="_blank" href="https://www.geeksforgeeks.org/sieve-of-eratosthenes/">link</a>.</p><p>There is no other previous knowledge required to understand what I'm about to say.</p><h2 id="two-algorithms-one-problem">Two algorithms, one problem</h2><p>The problem is to find all prime numbers from 0 (zero) to N. The kind of problems you learn to do when you are beginning your journey in programming. And I want to start with the simplest solution.</p><p>In the naive algorithm, we iterate over all numbers <code>x</code> from 2 to N. Then we check whether <code>x</code> has any divisor besides itself and one. For the last step, we can check for every number <code>d</code> between 2 and <code>x - 1</code> whether it is a divisor or not.</p><p>There is room for improvement in the last step because we only need to check the divisors that are less than or equal to the square root of x. A pseudocode of the algorithm is written below</p><pre><code> <span class="hljs-string">primes(N):</span> <span class="hljs-string">prime_numbers</span> <span class="hljs-string"><-</span> [] <span class="hljs-comment"># empty list</span> <span class="hljs-string">for</span> <span class="hljs-string">x</span> <span class="hljs-string">=</span> <span class="hljs-attr">2 to N:</span> <span class="hljs-string">is_prime</span> <span class="hljs-string"><-</span> <span class="hljs-literal">true</span> <span class="hljs-string">for</span> <span class="hljs-string">d</span> <span class="hljs-string">=</span> <span class="hljs-number">2</span> <span class="hljs-string">to</span> <span class="hljs-string">sqrt(x):</span> <span class="hljs-attr">if d divides x:</span> <span class="hljs-string">is_prime</span> <span class="hljs-string"><-</span> <span class="hljs-literal">false</span> <span class="hljs-comment"># we found a divisor of x so x is not a prime number</span> <span class="hljs-attr">if is_prime:</span> <span class="hljs-string">prime_numbers.add(x)</span> <span class="hljs-comment"># if we didn't find a divisor then x is prime</span> <span class="hljs-string">return</span> <span class="hljs-string">prime_numbers</span></code></pre><p>What's the runtime complexity of the previous algorithm? Well, we take every number from 2 to N and for each one, we iterate over all its possible divisors, so we make <code>O(N*sqrt(N))</code> operations, where <code>sqrt</code> stands for the square root function.</p><p>What about memory? We only store the primes we find. For a big enough N the amount of primes is relatively small compared with N. So, let's denote the memory complexity as <code>O(Primes(N))</code>, where <code>Primes(N)</code> is the number of primes between 0 and N. Note that this is optimum in terms of memory because we need at least to store all the prime numbers.</p><p>Probably you already know that there is a better algorithm to find all the prime numbers in a given range: The sieve of Eratosthenes. Let's look at it.</p><p>The idea is to maintain an array of length N where every entry is either false or true. If the i-th position in the array is true, then we say that <code>i</code> is a prime number, otherwise <code>i</code> is not prime.</p><p>We start with all positions with a true value and then, for every position, starting with <code>i = 2</code> we make false every multiple of <code>i</code>. We end up with an array in which every position with a true value represents a prime number. The code with some optimizations is presented below.</p><pre><code> primes(N): prime_numbers <- [] # empty list sieve <- a <span class="hljs-type">boolean</span> list <span class="hljs-keyword">with</span> length equal <span class="hljs-keyword">to</span> N <span class="hljs-keyword">and</span> <span class="hljs-keyword">all</span> its elements <span class="hljs-keyword">set</span> <span class="hljs-keyword">to</span> <span class="hljs-keyword">true</span> <span class="hljs-keyword">for</span> i = <span class="hljs-number">2</span> <span class="hljs-keyword">to</span> sqrt(N): # we <span class="hljs-keyword">only</span> need <span class="hljs-keyword">to</span> <span class="hljs-keyword">analyze</span> the numbers less than <span class="hljs-keyword">or</span> equal <span class="hljs-keyword">to</span> sqrt(N) <span class="hljs-keyword">if</span> sieve[i]: # <span class="hljs-keyword">if</span> sieve[i] <span class="hljs-keyword">is</span> <span class="hljs-keyword">true</span> <span class="hljs-keyword">then</span> i <span class="hljs-keyword">is</span> prime prime_numbers.<span class="hljs-keyword">add</span>(i) <span class="hljs-keyword">for</span> j = i*i <span class="hljs-keyword">to</span> N: # we can <span class="hljs-keyword">start</span> the <span class="hljs-keyword">inner</span> <span class="hljs-keyword">loop</span> <span class="hljs-keyword">in</span> i*i sieve[j] <- <span class="hljs-keyword">false</span> j <- j + i <span class="hljs-keyword">return</span> prime_numbers</code></pre><p>It can be proved that the algorithm described above does <a target="_blank" href="https://www.geeksforgeeks.org/how-is-the-time-complexity-of-sieve-of-eratosthenes-is-nloglogn/">O(N log(N))</a> operations which is better than the naive approach. Even it can be improved to <a target="_blank" href="https://www.geeksforgeeks.org/sieve-eratosthenes-0n-time-complexity">O(N)</a>! But we need to store an array with all the N numbers, and thus, we end up with a more expensive algorithm in memory terms. Here we have the trade-off: time in exchange for memory.</p><p>Can we design an algorithm that is faster than the naive idea but cheaper in memory terms than the sieve of Eratosthenes? </p><h2 id="dijkstra-and-the-production-line">Dijkstra and the production line</h2><p>In the '60s, E.W. Dijkstra wrote in one of his manuscripts [<a target="_blank" href="https://www.cs.utexas.edu/~EWD/ewd02xx/EWD249.PDF">PDF</a>] an algorithm that combines the naive and the sieve ideas. But before we talk about it, let's analyze the differences between the two previous algorithms. </p><p>When applying the naive algorithm we focus on analyzing whether every number is prime or not. When applying the sieve we analyze, for every prime number, all its multiples. The difference can be illustrated with the following analogy.</p><p>Imagine we want to build several products, like action figures. We have two options: building them one by one or applying a production line and in every stage, we add one component of the action figure. With the latter, we end up producing more in less time, but we need the "line infrastructure".</p><p>Dijkstra combined those ideas by analyzing a number at a time but taking advantage of the previous operations. We can maintain a pool of primes that have been already discovered, and, for each of those prime numbers store the least multiple that has not been analyzed yet. For example:</p><p>If we are analyzing number 6, then we have stored primes 2, 3, and 5, along with their least-not-analyzed-multiples that are: 6 for 2, also 6 for 3, and 10 for 5.</p><p>Then, when analyzing a new number we take the smallest multiple stored until that moment and if that multiple is greater than the new number, then we have found a new prime, otherwise, we have a composite number and we need to update the multiples of the stored primes that have the new number as its least multiple.</p><p>We begin storing the prime number <code>2</code> with the least multiple <code>4</code> as well. Then when analyzing <code>3</code> we find that <code>4 > 3</code> so <code>3</code> is prime. We store <code>3</code> along with its smallest multiple that has not been analyzed yet (<code>6</code>). When analyzing <code>4</code> we find that <code>4</code> is stored as a multiple of <code>2</code>, then we update the multiple of <code>2</code> that now will be <code>6</code>. When analyzing <code>5</code> we find that <code>6 > 5</code> so <code>5</code> is a prime number and we store it along with <code>10</code>, and so on... The code is presented below.</p><pre><code> <span class="hljs-string">primes(N):</span> <span class="hljs-string">primes_pool</span> <span class="hljs-string"><-</span> <span class="hljs-string">Heap(</span> <span class="hljs-string">(4,</span> <span class="hljs-number">2</span><span class="hljs-string">)</span> <span class="hljs-string">)</span> <span class="hljs-comment"># a heap that contains a tuple with a prime number (2) and its least multiple that hasn't been analyzed yet (4).</span> <span class="hljs-string">prime_numbers</span> <span class="hljs-string"><-</span> [<span class="hljs-number">2</span>] <span class="hljs-comment"># a list that contains the number 2</span> <span class="hljs-string">for</span> <span class="hljs-string">i</span> <span class="hljs-string">=</span> <span class="hljs-attr">3 to N:</span> <span class="hljs-string">tuple</span> <span class="hljs-string"><-</span> <span class="hljs-string">getMin(primes_pool)</span> <span class="hljs-comment"># get the tuple with the minimum multiple, the prime number attached to it is irrelevant</span> <span class="hljs-string">mult</span> <span class="hljs-string"><-</span> <span class="hljs-string">tuple.first</span> <span class="hljs-comment"># the multiple is stored in the first position of the tuple</span> <span class="hljs-string">if</span> <span class="hljs-string">mult</span> <span class="hljs-string">></span> <span class="hljs-attr">i:</span> <span class="hljs-string">prime_numbers.add(i)</span> <span class="hljs-comment"># i is prime!</span> <span class="hljs-string">primes_pool.insert(</span> <span class="hljs-string">(i*i,</span> <span class="hljs-string">i)</span> <span class="hljs-string">)</span> <span class="hljs-comment"># we insert i along with their least multiple, but it can be inserted i*i instead and the algorithm remains correct</span> <span class="hljs-string">continue</span> <span class="hljs-comment"># otherwise i isn't prime and then...</span> <span class="hljs-attr">for t such that t is in primes_pool and t.first is equal to mult:</span> <span class="hljs-string">t.first</span> <span class="hljs-string"><-</span> <span class="hljs-string">t.first</span> <span class="hljs-string">+</span> <span class="hljs-string">t.second</span> <span class="hljs-comment"># we update every tuple in the pool that has i as it's least multiple</span> <span class="hljs-string">return</span> <span class="hljs-string">prime_numbers</span></code></pre><p>The previous method only stores the prime numbers and another number for each of those primes. So we have a memory complexity of <code>O(Primes(N))</code> which is the same as the naive idea. If we store the prime numbers along with their multiples in a structure like a heap, we get a time complexity of <code>O(N*log N)</code> which is the same as the sieve! So we got what we wanted!</p><p>The trick here is that we don't need to mark every multiple of the given prime but just the least multiple.</p><p>I need to say that this is not a practical idea in the sense that the memory complexity of the sieve of Eratosthenes is not that bad and it is an algorithm very easy to implement. My point is that sometimes you have several ideas, each of them cannot be applied because of some flaw, then maybe is a good idea to combine their strengths and you can get a hybrid applicable solution for your problem. The Factory of Primes of Dijkstra has taught me to think in that way, although I have never implemented that algorithm in real-life scenarios. </p><h2 id="combining-arrays-with-linked-lists">Combining arrays with linked lists</h2><p>Arrays are simple structures that allow us to get an element by its index in constant time. But we need <code>O(N)</code> operations to insert or remove an element to/from the array in the worst case, where N is the length of the array.</p><p>On the other hand, linked lists are structures composed of nodes. Every node has a reference to the next one, and, in the case of doubly-linked lists, each node has also a reference to the previous one. In this case, we need <code>O(N)</code> operations to reach a node in the worst case, but the insert and remove operations can be done in constant time. I think it's natural to think about a "perfect data structure" that allows us to make the three operations in constant time. Sadly, such a structure does not exist, but I have found a middle point between the two opposite poles.</p><p>The "issue" with arrays is that they maintain a reference to all of their elements. That allows us to retrieve any element with the same amount of operations no matter where the element is. But maintaining those references is what makes the insertions and deletions so costly. When it comes to linked lists we only maintain a reference to the first and the last node and each node has a reference to its neighbors. So, when inserting or removing a new element we only need to change a few references. But that lack of references is the cause that we have to spend so many operations to get an element in the worst case.</p><p>Viewing the problem from that angle, the idea of finding a middle point in the number of references seems natural. What happens if we maintain a reference to <code>sqrt(N)</code> nodes of the linked list instead of referencing just the first and the last element?</p><p>That allows us to have a list of length <code>sqrt(N)</code> such that the distance between each of those nodes in the actual list is <code>sqrt(N)</code>. Having that, we can do each operation (index, insertion, and deletion) in <code>O(sqrt(N))</code>.</p><p>If you want to see more details about this structure and a Lisp implementation of it, you can do it <a target="_blank" href="https://github.com/josejorgers/indexed-linked-list">here</a>.</p><blockquote><p><strong>Update</strong>: I have written a post about this data structure. You can read it <a target="_blank" href="https://jj.hashnode.dev/improving-linked-lists">here</a></p></blockquote><h2 id="conclusions">Conclusions</h2><p>We have seen two examples of combining existing solutions to get another one that has some of the good qualities of each of the previous ones. My purpose was to show you this way of thinking, not just specific examples. </p><p>Note that in the case of Dijkstra's idea, we could achieve the time of the fastest solution and the memory complexity of the naive algorithm. In the second example, we just got a middle point, so the fastest solution is still faster and the memory-cheaper solution is still cheaper, but the new structure is like a decathlon athlete, it is good in every specificity, but not the best in anyone. So don't try to find the silver bullet, remember the no free lunch. Even Dijkstra's idea has the disadvantage of being harder to implement and to understand.</p><p>Hope you have learned or found attractive this post. Feel free to ask whatever you want or give me some feedback, I will appreciate that a lot. Stay tuned for the next post!</p>]]><![CDATA[The test, the bug and the hacker]]>https://blog.jayjaydev.com/the-test-the-bug-and-the-hackerhttps://blog.jayjaydev.com/the-test-the-bug-and-the-hackerSun, 27 Sep 2020 03:19:16 GMT<![CDATA[<p>Certain developer is creating the login page of a random website. When he finishes the work he proceeds to make a login attempt in his brand new page. There are things that don't work properly but he is able to fix them quickly. And finally, all users can enter with their accounts. Let's move this ticket to the Done board.</p><p>Once the project was finished it turned out that people couldn't log in with their accounts! Our developer was both stunned and sad at the same time. Luckily this was just a school project and wasn't a bank management software.</p><p>Have you experienced such situations? I certainly have. At some point in the development process he made some changes in the database structure, those changes made the login logic to fail. But he was logged in the page already and didn't remember to retest the login page. It was a hard to find error. Damn! </p><p>Does that mean that we are bad at making software? Yes, in a way.</p><h2 id="bugs-are-part-of-the-game">Bugs are part of the game</h2><p>Yes, coding that way is certainly wrong. But is not the error itself what was wrong. We are all humans, we make mistakes in every activity we carry on and coding is not an exception. Bugs are part of the game. </p><p>What we need to do is to be prepared to detect that error as soon as possible. We cannot avoid bugs, but we can make a lot of things instead of just login once or twice in order to detect bugs.</p><blockquote><p>Hope at this point you are motivated enough to keep reading about how to get rid of bugs. But maybe you need to know some examples and figures of how harmful bugs can be. I recommend you to take a look at this <a target="_blank" href="https://www.softwaretestingnews.co.uk/the-real-cost-of-software-bugs/">page</a></p></blockquote><h2 id="we-need-to-test">We need to test!</h2><p>There will be bugs in the code we'll write tomorrow. Why not take care about the "pesticide" today? So, start the project that will test your actual project right now!</p><p>There are several ways testing can be carried on, and there are several kinds of tests as well. This post is not about the characteristics of each type of test. Instead I want to make a shallow introduction about the necessity of testing and the right way to test.</p><p>Companies that are large enough have entire departments of Quality Assurance (QA) for the software they produce. Testing projects are comparable with the actual product in both length and resources consumption. So yes, you maybe will need to code a lot for the sole purpose of "capturing" your bugs. But it's worth every second and line of code when you are working on a complex enough project.</p><p>A bug is harmful only if it finds its way to the released product. The cost of fixing a bug decreases dramatically when it is discovered in an early stage.</p><p>Although testing can reduce cost a lot, it seems kind of expensive as well. Yes, I know it is order of magnitude less expensive. But how to make it cheaper? What is the right way to go?</p><h2 id="the-way-of-testing">The way of testing</h2><p>The first and most important thing to take into account is that testing <strong>can detect the presence of bugs but not their absence</strong>. So you need to focus your testing on finding bugs, instead of trying to prove the correctness of the software.</p><p>In order to be relatively easy to test, your project should be built according with good practices and principles. Actually testing is an effective way to find the best architecture and design patterns. Yeah, you should test even in the earliest stages of the project! Your software should be divided in several modules that fulfill very specific requirements, and we need to be able to test those modules either in isolation or interacting with each other.</p><p>We need to automatize as much as we can. And the scripts that contains tests should be light enough to be run lot of times with nearly no cost. Every task that is carried on by humans is prone to errors and is (at least in this context) more expensive. But it might happen that our software would require some test that can't be automated or that would be more expensive if we automatize it, due to the necessity of constant maintenance that it would need, among other possible issues.</p><p>But then... Don't we need a test to find bugs on the test itself?</p><h2 id="the-snake-wont-eat-its-tail">The snake won't eat its tail</h2><p>Testing is not bug free. You can certainly introduce some bugs when coding an automated test. Then it seems natural to think about a test that tests the test. But then we'll need a test that tests the test that tests the test. And of course we can come up with the idea that testing is a fruitless effort since it tries to capture some bugs while introducing some others.</p><p>Don't worry. I don't like to do fruitless work either, and making this post would be so if that last idea was true. </p><p>I just want to mention another important requirement that every automated test should fulfill: <strong>it needs to be simple</strong>. </p><p>Every complex logic needs to be out of the test and, potentially, be tested by a short, readable and simple piece of code</p><p>Thus, test will always be almost error free. Readability is an important feature as well since it allows to find any possible bug introduced in the test very quickly.</p><h2 id="conclusions">Conclusions</h2><p>This post is an overview of testing as a necessity in the software development. I approached the subject as an effective way to discover errors during the software construction, which is dramatically cheaper than discovering them when the harm is done. But testing is more than that, it is the most effective way to build good quality software. It should be present in every stage of the development and even be in charge of every stage. Testing is the heart of the software quality assurance.</p><p>As important as testing is testing in the right way. So you need your test to be simple, short and readable. That last feature is an important one! Not just because the reason mentioned above but because a test suite can represent the requirements of your product. Yes, you can "define" your product by testing it, but just if that definition can be easily read. </p><p>This post is far from being a deep guide for testing or a treaty about every single aspect of testing. My sole purpose is to motivate you to learn about it. I hope this requirement was fulfilled. Thanks for reading</p>]]><![CDATA[<p>Certain developer is creating the login page of a random website. When he finishes the work he proceeds to make a login attempt in his brand new page. There are things that don't work properly but he is able to fix them quickly. And finally, all users can enter with their accounts. Let's move this ticket to the Done board.</p><p>Once the project was finished it turned out that people couldn't log in with their accounts! Our developer was both stunned and sad at the same time. Luckily this was just a school project and wasn't a bank management software.</p><p>Have you experienced such situations? I certainly have. At some point in the development process he made some changes in the database structure, those changes made the login logic to fail. But he was logged in the page already and didn't remember to retest the login page. It was a hard to find error. Damn! </p><p>Does that mean that we are bad at making software? Yes, in a way.</p><h2 id="bugs-are-part-of-the-game">Bugs are part of the game</h2><p>Yes, coding that way is certainly wrong. But is not the error itself what was wrong. We are all humans, we make mistakes in every activity we carry on and coding is not an exception. Bugs are part of the game. </p><p>What we need to do is to be prepared to detect that error as soon as possible. We cannot avoid bugs, but we can make a lot of things instead of just login once or twice in order to detect bugs.</p><blockquote><p>Hope at this point you are motivated enough to keep reading about how to get rid of bugs. But maybe you need to know some examples and figures of how harmful bugs can be. I recommend you to take a look at this <a target="_blank" href="https://www.softwaretestingnews.co.uk/the-real-cost-of-software-bugs/">page</a></p></blockquote><h2 id="we-need-to-test">We need to test!</h2><p>There will be bugs in the code we'll write tomorrow. Why not take care about the "pesticide" today? So, start the project that will test your actual project right now!</p><p>There are several ways testing can be carried on, and there are several kinds of tests as well. This post is not about the characteristics of each type of test. Instead I want to make a shallow introduction about the necessity of testing and the right way to test.</p><p>Companies that are large enough have entire departments of Quality Assurance (QA) for the software they produce. Testing projects are comparable with the actual product in both length and resources consumption. So yes, you maybe will need to code a lot for the sole purpose of "capturing" your bugs. But it's worth every second and line of code when you are working on a complex enough project.</p><p>A bug is harmful only if it finds its way to the released product. The cost of fixing a bug decreases dramatically when it is discovered in an early stage.</p><p>Although testing can reduce cost a lot, it seems kind of expensive as well. Yes, I know it is order of magnitude less expensive. But how to make it cheaper? What is the right way to go?</p><h2 id="the-way-of-testing">The way of testing</h2><p>The first and most important thing to take into account is that testing <strong>can detect the presence of bugs but not their absence</strong>. So you need to focus your testing on finding bugs, instead of trying to prove the correctness of the software.</p><p>In order to be relatively easy to test, your project should be built according with good practices and principles. Actually testing is an effective way to find the best architecture and design patterns. Yeah, you should test even in the earliest stages of the project! Your software should be divided in several modules that fulfill very specific requirements, and we need to be able to test those modules either in isolation or interacting with each other.</p><p>We need to automatize as much as we can. And the scripts that contains tests should be light enough to be run lot of times with nearly no cost. Every task that is carried on by humans is prone to errors and is (at least in this context) more expensive. But it might happen that our software would require some test that can't be automated or that would be more expensive if we automatize it, due to the necessity of constant maintenance that it would need, among other possible issues.</p><p>But then... Don't we need a test to find bugs on the test itself?</p><h2 id="the-snake-wont-eat-its-tail">The snake won't eat its tail</h2><p>Testing is not bug free. You can certainly introduce some bugs when coding an automated test. Then it seems natural to think about a test that tests the test. But then we'll need a test that tests the test that tests the test. And of course we can come up with the idea that testing is a fruitless effort since it tries to capture some bugs while introducing some others.</p><p>Don't worry. I don't like to do fruitless work either, and making this post would be so if that last idea was true. </p><p>I just want to mention another important requirement that every automated test should fulfill: <strong>it needs to be simple</strong>. </p><p>Every complex logic needs to be out of the test and, potentially, be tested by a short, readable and simple piece of code</p><p>Thus, test will always be almost error free. Readability is an important feature as well since it allows to find any possible bug introduced in the test very quickly.</p><h2 id="conclusions">Conclusions</h2><p>This post is an overview of testing as a necessity in the software development. I approached the subject as an effective way to discover errors during the software construction, which is dramatically cheaper than discovering them when the harm is done. But testing is more than that, it is the most effective way to build good quality software. It should be present in every stage of the development and even be in charge of every stage. Testing is the heart of the software quality assurance.</p><p>As important as testing is testing in the right way. So you need your test to be simple, short and readable. That last feature is an important one! Not just because the reason mentioned above but because a test suite can represent the requirements of your product. Yes, you can "define" your product by testing it, but just if that definition can be easily read. </p><p>This post is far from being a deep guide for testing or a treaty about every single aspect of testing. My sole purpose is to motivate you to learn about it. I hope this requirement was fulfilled. Thanks for reading</p>]]>