<?xml version="1.0" encoding="UTF-8"?>
<rss  xmlns:atom="http://www.w3.org/2005/Atom" 
      xmlns:media="http://search.yahoo.com/mrss/" 
      xmlns:content="http://purl.org/rss/1.0/modules/content/" 
      xmlns:dc="http://purl.org/dc/elements/1.1/" 
      version="2.0">
<channel>
<title>Jian Wei Cheong</title>
<link>https://jianweicheong.codeberg.page/posts.html</link>
<atom:link href="https://jianweicheong.codeberg.page/posts.xml" rel="self" type="application/rss+xml"/>
<description>Personal page of Jian Wei Cheong</description>
<generator>quarto-1.9.37</generator>
<lastBuildDate>Sat, 10 Oct 2026 16:00:00 GMT</lastBuildDate>
<item>
  <title>Using game engines (Bevy) for scientific simulations?</title>
  <link>https://jianweicheong.codeberg.page/posts/game-engine-simulation.html</link>
  <description><![CDATA[ 




<section id="out-with-the-old" class="level1 page-columns page-full">
<h1>Out with the old</h1>
<p>Running simulations is part and parcel of being a scientist.</p>
<p>Most simulations are simply numerical computations; run it once and plot some results. For this, scientists have long relied on Python, MATLAB, R, Julia, or Fortran, C, and C++ for the more computationally expensive stuff.</p>
<p>These have served me well too, and if I require visualizations beyond simple plots, they typically covers them too (e.g., left video below).</p>
<div class="quarto-layout-panel" data-layout="[1.125,2]" data-fig-align="bottom">
<div class="quarto-layout-row">
<div class="quarto-layout-cell" style="flex-basis: 36.0%;justify-content: flex-start;">
<div class="quarto-figure quarto-figure-center">
<figure class="figure">
<p><video src="../media/posts/game-engine-simulation/bus.mp4" class="img-fluid" loop="true" autoplay="true" controls=""><a href="../media/posts/game-engine-simulation/bus.mp4">Video</a></video></p>
<figcaption>Simulation of NTU’s campus bus route with <a href="https://juliadynamics.github.io/Agents.jl/stable/">Agents.jl</a> in Julia.</figcaption>
</figure>
</div>
</div>
<div class="quarto-layout-cell" style="flex-basis: 64.0%;justify-content: flex-start;">
<div class="quarto-figure quarto-figure-center">
<figure class="figure">
<p><video src="../media/posts/game-engine-simulation/ratchet_render.mp4" class="img-fluid" loop="true" autoplay="true" controls=""><a href="../media/posts/game-engine-simulation/ratchet_render.mp4">Video</a></video></p>
<figcaption>Visualization of an information ratchet-tape system <span class="citation" data-cites="HeInformation2022">(He et al. 2022)</span> in <a href="https://www.blender.org/">Blender</a>.</figcaption>
</figure>
</div>
</div>
</div>
</div>
<p>Recently, I’ve also been experimenting with <a href="https://www.blender.org/">Blender</a> as a visualization tool. Thanks to <a href="https://docs.blender.org/api/current/index.html">its integration with Python</a>, one can perform all their computations in Python, and simply “plot” them in Blender<sup>1</sup>.</p>
<div class="no-row-height column-margin column-container"><div id="fn1"><p><sup>1</sup>&nbsp;See also <a href="../posts/betty-hill-star-map.html">this post</a> where I plotted our local stellar neighborhood with Blender. <video src="../media/posts/betty-hill-star-map/30pc_spin_fade_connections.webm" class="img-fluid" loop="true" autoplay="true" controls=""><a href="../media/posts/betty-hill-star-map/30pc_spin_fade_connections.webm">Video</a></video></p></div></div><p>Of course, even without Python, Blender is quite powerful in simulating physical systems, especially with <a href="https://docs.blender.org/manual/en/latest/modeling/geometry_nodes/introduction.html">geometry nodes</a>. Combined with its beautiful rendering capabilities with <a href="https://www.blender.org/features/rendering/">Cycles</a>, it is great for scientific visualization (e.g., right video above).</p>
<p>Still, despite my love for Blender, I find the hybrid Python + Blender workflow to be a little clunky, and I’ve never really been fond of Python for running simulation either.</p>
<p>So what are the alternatives<sup>2</sup>?</p>
<div class="no-row-height column-margin column-container"><div id="fn2"><p><sup>2</sup>&nbsp;And I’m not ready for sorcery like <a href="https://www.opengl.org/">OpenGL</a> or <a href="https://www.raylib.com/">raylib</a> yet.</p></div></div><p>Videos games are some of the most complex world simulations we have. Complex system scientists run <a href="https://en.wikipedia.org/wiki/Cellular_Automaton">cellular automata</a> to study the science of complexity, all while there’s games like <a href="https://www.bay12games.com/dwarves/">Dwarf Fortress</a>, which is arguably the most complex cellular automaton-type simulation we have.</p>
<p>So why not use a game engine?</p>
<p>For one, I still have the same problem with the Python + Blender workflow, except this time it’s C# + <a href="https://unity.com/">Unity</a>, C++ + <a href="https://www.unrealengine.com/">Unreal</a>, or GDScript + <a href="https://godotengine.org/">Godot</a>.</p>
<p>Thankfully though, there’s <a href="https://bevy.org/">Bevy</a>.</p>
</section>
<section id="in-with-the-new-bevy" class="level1 page-columns page-full">
<h1>In with the new: Bevy</h1>
<div class="quarto-figure quarto-figure-center">
<figure class="figure">
<p><video src="../media/posts/game-engine-simulation/flocking.webm" class="img-fluid" loop="true" autoplay="true" controls=""><a href="../media/posts/game-engine-simulation/flocking.webm">Video</a></video></p>
<figcaption>A bevy of “boid” with flocking behaviors (<a href="https://en.wikipedia.org/wiki/Boids">Boids</a>) running with <a href="https://bevy.org/">Bevy</a> in Rust in real-time.</figcaption>
</figure>
</div>
<p>Released in 2020, <a href="https://bevy.org">Bevy</a> is a relatively new free and open source game engine in Rust. It is also quite experimental, having breaking changes every few releases. Yet, it has produced fantastic and beautiful games like <a href="https://en.wikipedia.org/wiki/Tiny_Glade">Tiny Glade</a>, <a href="https://store.steampowered.com/app/4410710/Fields_of_Aaru/">Fields of Aaru</a>, <a href="https://store.steampowered.com/app/2628450/Times_of_Progress/">Times of Progress</a>, and many more.</p>
<p>More importantly for me, unlike other game engines, Bevy require no GUI or visual editor<sup>3</sup>. This means that the workflow is similar to traditional computations that scientists are familiar with, where everything is just code.</p>
<div class="no-row-height column-margin column-container"><div id="fn3"><p><sup>3</sup>&nbsp;Though it is a <a href="https://github.com/bevyengine/bevy/discussions/7100">planned feature</a>, and there are already alternatives like <a href="https://github.com/jbuehler23/jackdaw">Jackdaw</a>.</p></div></div><p>In addition to <a href="https://benchmarksgame-team.pages.debian.net/benchmarksgame/box-plot-summary-charts.html">its speed</a>, Rust is also a joy to write in with type and memory safeties, and one can achieve beautiful renders and visualizations thanks to Bevy’s ray-tracer <a href="https://jms55.github.io/posts/2026-09-18-solari-bevy-0-20/">Solari</a>.</p>
<section id="interactivity" class="level2 page-columns page-full">
<h2 class="anchored" data-anchor-id="interactivity">Interactivity</h2>
<p>You might have already noticed from the video above, being a game engine, the user can interact with their simulation freely in real-time. This is especially great when exploring the system that you are simulating.</p>
<p>In fact, just imagine combining any feature of a video game into your scientific simulation:</p>
<ul>
<li>Freely changing the initial conditions</li>
<li>Freely changing the position or properties of an agent that you are simulating live</li>
<li>Saving and loading different states of the simulation</li>
<li>Seeing the plots of some measures live</li>
</ul>
<p>I have always dreamed of a more efficient and convenient “UX” for scientific computational research.</p>
<p>How many hundreds of times have you gone through the same old boring workflow of writing the same templates of codes, looping over multiple initial conditions, keeping track of quantities that you’re concerned about, running them, plotting the results, rinse and repeat<sup>4</sup>?</p>
<div class="no-row-height column-margin column-container"><div id="fn4"><p><sup>4</sup>&nbsp;I’ve grown quite weary of writing the thousandth quantum system in Python/Julia that I’m genuinely making (albeit slowly) an interactive general open quantum system builder in Bevy.</p></div></div><p>And consider all the times in a group meeting where all we can show is some pre-made plots, and all we can say is “I’ll go back and check” when asked about what would happen if we changed something in our simulation?</p>
<p>By spending some extra time and effort on coding upfront, we can put all of this behind us<sup>5</sup>.</p>
<div class="no-row-height column-margin column-container"><div id="fn5"><p><sup>5</sup>&nbsp;If you are doing serious investigations into a particular system rather than just one-off numerical computations, it’s probably worth it to spend the time and effort.</p></div></div></section>
<section id="ecs-the-natural-abm" class="level2 page-columns page-full">
<h2 class="anchored" data-anchor-id="ecs-the-natural-abm">ECS, the natural ABM?</h2>
<p>The bus route and flocking simulations above are what’s called an <a href="https://en.wikipedia.org/wiki/Agent-based_model">agent-based model (ABM)</a>, where a collection of autonomous agents or entities have certain behaviors, and react with their environment and one another under some rules<sup>6</sup>.</p>
<div class="no-row-height column-margin column-container"><div id="fn6"><p><sup>6</sup>&nbsp;Agents.jl has a <a href="https://juliadynamics.github.io/Agents.jl/stable/comparison/#Table-based-comparison">table of comparison</a> between some popular ABM frameworks.</p></div></div><p>If you have even just one inkling of what a video game is, this sounds rather like one doesn’t it?</p>
<p>Almost all games employ some version of this, and games that exploit this feature into gameplay are often called <em>systemic games</em><sup>7</sup>, where almost all systems inside the game interacts with one another, either to create a simulation, or to generate emergent gameplay.</p>
<div class="no-row-height column-margin column-container"><div id="fn7"><p><sup>7</sup>&nbsp;<a href="https://www.youtube.com/watch?v=SnpAAX9CkIc">This video</a> by Game Maker’s Toolkit is a nice summary of systemic games.</p></div></div><p>This kind of games are especially suitable for a kind of programming paradigm called <a href="https://en.wikipedia.org/wiki/Entity_component_system">entity component system (ECS)</a>, that Bevy is based on.</p>
<p>Let’s look at the flocking simulation as an example.</p>
<table class="caption-top table">
<colgroup>
<col style="width: 33%">
<col style="width: 33%">
<col style="width: 33%">
</colgroup>
<thead>
<tr class="header">
<th>Flocking simulation</th>
<th>ABM (object-oriented)</th>
<th>ECS (data-oriented)</th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td>A bird</td>
<td>Agent object</td>
<td>Entity + Component</td>
</tr>
<tr class="even">
<td>Bird’s position &amp; velocity</td>
<td>State variables of agent object</td>
<td>Component data</td>
</tr>
<tr class="odd">
<td>Boid behaviors (separation, alignment, cohesion)</td>
<td>Agent object’s methods</td>
<td>System functions acting on the components of entities</td>
</tr>
<tr class="even">
<td>Simulation progression</td>
<td>Loop through each agent</td>
<td>World tick</td>
</tr>
</tbody>
</table>
<p><br>
</p>
<p>In the traditional way to implement ABM (object-oriented), each agent carries its own states and update functions, i.e., each agent “knows what to do” in the simulation like an actual bird.</p>
<p>On the other hand, in ECS (data-oriented/driven), an agent is an entity that is assigned some components, and is defined/identified by what components it is assigned. What we call a “bird” is just an ID attached with the components of position and velocity.</p>
<p>Perhaps one could frame this philosophically: In the data-oriented ECS, a bird is only considered a bird if it has a position and velocity, while in the object-oriented ABM, a bird is a bird by definition.</p>
<p>Furthermore, the ECS bird has to rely on the “world” or system to update its components, i.e., each agent “does not know what to do” in the simulation. The “world” will look at all entities who has certain components and whose components fulfill some conditions, and updates them accordingly.</p>
<p>This is like if reality suddenly dictates that “everyone who wears glasses and live in Singapore are now taller by 1cm”, and updates them accordingly (ECS), as compared to each human having a secret “power” in them that once these conditions are fulfilled, will grow taller by 1cm (ABM).</p>
<p>This somewhat less intuitive way of doing things turns out to have a lot of advantages in terms of scalability, parallelism, modularity, and performance <span class="citation" data-cites="TasnimEssence2026">(Tasnim and Zhao 2026)</span>, and is becoming more and more popular.</p>
</section>
<section id="no-limits" class="level2 page-columns page-full">
<h2 class="anchored" data-anchor-id="no-limits">No limits</h2>
<p>Finally, as Rust is a general-purpose programming language, there really isn’t any limits to the kind of simulations you can run, especially with its <a href="https://crates.io/">huge ecosystem of Crates</a>.</p>
<p>There’s <a href="https://docs.rs/ndarray/latest/ndarray/"><code>ndarray</code></a> for high-dimensional arrays, <a href="https://nalgebra.rs/"><code>nalgebra</code></a> or <a href="https://faer.veganb.tw/"><code>faer</code></a> for linear algebra, <a href="https://burn.dev/"><code>burn</code></a> or <a href="https://github.com/huggingface/candle"><code>candle</code></a> for machine learning, <a href="https://github.com/georust/geo"><code>geo</code></a> for geospatial computing, and of course <a href="https://pola.rs/"><code>polars</code></a>, the superfast dataframe that is supplanting the popular <a href="https://pandas.pydata.org/"><code>pandas</code></a> in Python, thanks to tools like <a href="https://pyo3.rs/"><code>pyo3</code></a> and <a href="https://www.maturin.rs/"><code>maturin</code></a> that allow the calling of Rust libraries in Python.</p>
<p>You will write your scientific computations in Rust with memory safety, <a href="https://doc.rust-lang.org/book/ch16-00-concurrency.html">fearless concurrency</a>, modern features, and huge performance gains <span class="citation" data-cites="VeytsmanRewrite2024">(Veytsman et al. 2024)</span>, while Bevy is just another crate that will handle the rendering, interactivity, and ECS.</p>
<p>Many people <a href="https://github.com/bevyengine/bevy/discussions/1678">also had the same idea</a> to use Bevy beyond game development<sup>8</sup>, <a href="https://github.com/cuppachino/bevy_logic">logic gate simulator</a>, <a href="https://github.com/biosustain/shu">app to plot metabolic map</a>, <a href="https://mo8it.com/blog/physics-simulations-in-bevy/">simulating rigid bodies</a>, <a href="https://www.fslabs.ca/">visualizing spatial data</a>, <a href="https://github.com/AOS55/bevy-sph">fluid simulation</a>, and even <a href="https://github.com/TeamAtomECS/AtomECS/">cold atom simulations</a> are some examples.</p>
<div class="no-row-height column-margin column-container"><div id="fn8"><p><sup>8</sup>&nbsp;There’s a <a href="https://www.youtube.com/watch?v=XyXMKuclTcQ">great talk</a> by <a href="https://github.com/alice-i-cecile">Alice Cecile</a> at the Scientific Computing for Rust 2025 workshop, who’s the maintainer of Bevy and who used to be a complex system ecologist.</p></div><div id="fn9"><p><sup>9</sup>&nbsp;Bevy is also cross platform, allowing one to run their interactive simulations on Windows, MacOS, Linux, Android, iOS, and even the web with WebGL2 or WebGPU. See some examples of running on the web <a href="https://bevy.org/examples/">here</a>.</p></div></div><p>With Bevy, computational scientists can gain a fundamentally different workflow where interactivity becomes native to their simulations, beyond the same old run and wait cycle<sup>9</sup>.</p>
<p>As a final demonstration, here’s a simple three-body interaction rendered in Bevy with around 200 lines of code, demonstrating that it also has the potential for just scientific visualization like Blender.</p>
<div class="quarto-figure quarto-figure-center">
<figure class="figure">
<p><video src="../media/posts/game-engine-simulation/3body.webm" class="img-fluid" loop="true" autoplay="true" controls=""><a href="../media/posts/game-engine-simulation/3body.webm">Video</a></video></p>
<figcaption>Three-body interaction in Bevy.</figcaption>
</figure>
</div>



</section>
</section>


<a onclick="window.scrollTo(0, 0); return false;" id="quarto-back-to-top"><i class="bi bi-arrow-up"></i> Back to top</a><div id="quarto-appendix" class="default"><section class="quarto-appendix-contents" id="quarto-bibliography"><h2 class="anchored quarto-appendix-heading">References</h2><div id="refs" class="references csl-bib-body hanging-indent" data-entry-spacing="0">
<div id="ref-HeInformation2022" class="csl-entry">
He, Lianjie, Andri Pradana, Jian Wei Cheong, and Lock Yue Chew. 2022. <span>“Information Processing Second Law for an Information Ratchet with Finite Tape.”</span> <em>Phys. Rev. E</em> 105 (May): 054131. <a href="https://doi.org/10.1103/PhysRevE.105.054131">https://doi.org/10.1103/PhysRevE.105.054131</a>.
</div>
<div id="ref-TasnimEssence2026" class="csl-entry">
Tasnim, Anisha, and Tian Zhao. 2026. <span>“The Essence of Entity Component System.”</span> In <em>Proceedings of the 41st ACM/SIGAPP Symposium on Applied Computing</em>, 1320–29. SAC ’26. ACM. <a href="https://doi.org/10.1145/3748522.3779910">https://doi.org/10.1145/3748522.3779910</a>.
</div>
<div id="ref-VeytsmanRewrite2024" class="csl-entry">
Veytsman, Willow, Shuang Zhai, Chen Ding, and Adam B. Sefkow. 2024. <span>“Rewrite It in Rust: A Computational Physics Case Study.”</span> <a href="https://arxiv.org/abs/2410.19146">https://arxiv.org/abs/2410.19146</a>.
</div>
</div></section></div> ]]></description>
  <category>programming</category>
  <category>software</category>
  <category>physics</category>
  <guid>https://jianweicheong.codeberg.page/posts/game-engine-simulation.html</guid>
  <pubDate>Sat, 10 Oct 2026 16:00:00 GMT</pubDate>
</item>
<item>
  <title>Can you teach programming in 15 minutes?</title>
  <link>https://jianweicheong.codeberg.page/posts/program-15-min.html</link>
  <description><![CDATA[ 




<blockquote class="blockquote">
<p><em>“You think you know when you learn, are more sure when you can write, even more when you can teach, but certain when you can program.”</em></p>
<div style="text-align: right;">
<p>— <a href="https://en.wikipedia.org/wiki/Alan_Perlis">Alan Perlis</a> <br> First recipient of the Turing Award</p>
</div>
</blockquote>
<p>For three years now I have conducted a programming class to high-school level students as part of my school’s annual outreach program, teaching them basic Python to compute and plot the <a href="https://en.wikipedia.org/wiki/Logistic_map">logistic map</a> in a two hours class. With six classes per year for three years, that’s 36 hours of classes, surprisingly more than a year of teaching tutorial classes.</p>
<p>With the advent of vibecoding though, this is the first year where I have to begin with the quote above, in an attempt to motivate the students into learning how to program themselves. Though, I’m sure the cool visualizations and simulations that I showed did much of the heavylifting.</p>
<p>The students didn’t sign up for a programming class afterall. Instead, they signed up for a three days camp to experience what doing a Physics degree is like. In the three days, they participated in various lab activities, doing cool experiments with lasers or fabricating a solar cell etc.</p>
<p>And then they come to my class and have to endure sitting in front of a computer for two hours, all while getting thrown errors from Python. So we already lost half the battle here.</p>
<p>Of course, most have no programming experience, and the actual teaching of basic Python programming spans just around <strong>15 minutes</strong>, where they follow along a haphazard explanation about arrays, for loops, plotting, or whatnot in a Jupyter notebook, but with as little computing jargon as possible.</p>
<p>Afterwards, we let go and let them apply what they learned in the 15 minutes to solve a few simple tasks. A couple of TAs and me will then walk around and guide them if they encounter any problems. “Hands-on” is the point of their three days camp afterall.</p>

<div class="no-row-height column-margin column-container"><div class="margin-aside">
<p>Here’s the breakdown of the two hours:</p>
<ol type="1">
<li>Introduction and theory (15 minutes)</li>
<li>Learning Python (15 minutes)</li>
<li>Doing the tasks (1.5 hours)</li>
</ol>
</div></div><section id="the-tasks" class="level1 page-columns page-full">
<h1>The tasks</h1>
<p>Their tasks are quite simple really.</p>
<p>After some introduction about the idea of population dynamics and chaos at the start of the class, we got them to derive the logistic map:</p>
<p><img src="https://latex.codecogs.com/png.latex?%0Ax_%7Bn+1%7D%20=%20Ax_n(1-x_n),%0A"> where <img src="https://latex.codecogs.com/png.latex?0%20%5Cleq%20x_n%20%5Cleq%201"> and <img src="https://latex.codecogs.com/png.latex?0%20%5Cleq%20A%20%5Cleq%204">.</p>
<p><img src="https://latex.codecogs.com/png.latex?x_n"> models a proportion of population at year <img src="https://latex.codecogs.com/png.latex?n">, and <img src="https://latex.codecogs.com/png.latex?A"> is a parameter that might represents some environmental conditions. Depending on the values of <img src="https://latex.codecogs.com/png.latex?A">, the population can rise and fall, or exhibit complex behaviors.</p>
<section id="first-task" class="level2 page-columns page-full">
<h2 class="anchored" data-anchor-id="first-task">First task</h2>
<p>Their first task is to plot <img src="https://latex.codecogs.com/png.latex?x_n"> against <img src="https://latex.codecogs.com/png.latex?n"> for some specified values of <img src="https://latex.codecogs.com/png.latex?A"> and an arbitrary <img src="https://latex.codecogs.com/png.latex?x_0">.</p>
<p>After much iteration, we decided that the simplest solution is to make use of Numpy’s arrays:</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb1" style="background: #f1f3f5;"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb1-1"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">import</span> numpy <span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">as</span> np</span>
<span id="cb1-2"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">import</span> matplotlib.pyplot <span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">as</span> plt</span>
<span id="cb1-3"></span>
<span id="cb1-4">x <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> np.zeros(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">50</span>)</span>
<span id="cb1-5">x[<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0.2</span></span>
<span id="cb1-6"></span>
<span id="cb1-7">A <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0.5</span></span>
<span id="cb1-8"><span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">for</span> n <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">in</span> <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">range</span>(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">49</span>):</span>
<span id="cb1-9">    x[n<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> A <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> x[n] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> (<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span> x[n])</span>
<span id="cb1-10"></span>
<span id="cb1-11">plt.plot(x)</span></code></pre></div></div>

<div class="no-row-height column-margin column-container"><div class="margin-aside">
<p>The students actually worked on multiple questions with different values of <img src="https://latex.codecogs.com/png.latex?A">, and explanations were given in between to allow them to appreciate and understand what the plots are showing, e.g., <em>fixed points</em>, <em>stability</em>, <em>period doubling</em>, <em>onset of chaos</em> etc.</p>
</div></div><p>where we have taken <img src="https://latex.codecogs.com/png.latex?A%20=%200.5"> and <img src="https://latex.codecogs.com/png.latex?x_0%20=%200.2">.</p>
<p>Other than the <code>np.zeros(50)</code> and the <code>for</code> loop, this translates directly to the equation, with <code>x[0]</code> being <img src="https://latex.codecogs.com/png.latex?x_0">, and <code>x[n+1] = A * x[n] * (1 - x[n])</code> being <img src="https://latex.codecogs.com/png.latex?x_%7Bn+1%7D%20=%20Ax_n(1%20-%20x_n)">.</p>
<p>We made sure to teach more than just what are needed in the solutions though, so that the students can try to come up with their own solutions.</p>
</section>
<section id="second-task" class="level2">
<h2 class="anchored" data-anchor-id="second-task">Second task</h2>
<p>The second task is more challenging; they are to plot a <a href="https://en.wikipedia.org/wiki/Bifurcation_diagram">bifurcation diagram</a> for a range of <img src="https://latex.codecogs.com/png.latex?A">.</p>
<p>Again, we tried our best to come up with what we think is the simplest solution of</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb2" style="background: #f1f3f5;"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb2-1"><span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">for</span> A <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">in</span> np.linspace(<span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">3.4</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">4</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">3000</span>):</span>
<span id="cb2-2">    x <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> np.zeros(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">600</span>)</span>
<span id="cb2-3">    x[<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0.2</span></span>
<span id="cb2-4"></span>
<span id="cb2-5">    <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">for</span> n <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">in</span> <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">range</span>(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">599</span>):</span>
<span id="cb2-6">        x[n<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> A <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> x[n] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> (<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span> x[n])</span>
<span id="cb2-7"></span>
<span id="cb2-8">    plt.plot([A] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">300</span>, x[<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">300</span>:], <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">','</span>)</span>
<span id="cb2-9"></span>
<span id="cb2-10">plt.xlabel(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"$A$"</span>)</span>
<span id="cb2-11">plt.ylabel(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"$x_n$"</span>)</span></code></pre></div></div>
<p>Quite simply, we repeat the first task but for different values of <img src="https://latex.codecogs.com/png.latex?A">, and plot multiple values of <img src="https://latex.codecogs.com/png.latex?x_n"> for each <img src="https://latex.codecogs.com/png.latex?A">. The only caveat is that we only want to plot for large values of <img src="https://latex.codecogs.com/png.latex?n"> so that the population has time to reach its long-time behavior for each <img src="https://latex.codecogs.com/png.latex?A">. This is the reason for the <code>x[300:]</code>.</p>
<p>The result is quite spectacular. It captures the <em>period-doubling route to chaos</em>, <em>intermittency route to chaos</em>, <em>islands of stability</em>, the <em>fractal</em> nature of the map etc., demonstrating that such a simple equation can lead to complex and chaotic behaviors. Imagine how the actual and much more complex evolution of real-world populations would look like!</p>
<div class="quarto-figure quarto-figure-center">
<figure class="figure">
<p><img src="https://jianweicheong.codeberg.page/media/posts/program-15-min/demo2.png" class="img-fluid figure-img"></p>
<figcaption>Bifurcation diagram of the logistic map.</figcaption>
</figure>
</div>
</section>
</section>
<section id="what-students-struggle-with" class="level1">
<h1>What students struggle with</h1>
<p>This is like 16 lines of Python code (four are the same for both tasks), and Python is widely touted as one of the easiest to learn programming language.</p>
<p>So this shouldn’t be too difficult right?</p>
<p>Here are some difficulties that the students struggled with that I noticed:</p>
<ol type="1">
<li><p><strong>Not understanding or forgetting about indentation.</strong> Python’s supposed simplicity of replacing braces with indentations seems to backfire here.</p></li>
<li><p><strong>Missing colons for the <code>for</code> loop.</strong> The colon is quite elusive and easy to forget.</p></li>
<li><p><strong>Confusing <code>[]</code> and <code>()</code>.</strong> Though this will never convince me that MATLAB’s choice of using <code>()</code> for both function calls and array indexing is sane.</p></li>
<li><p><strong>The need to initialize arrays is quite arcane.</strong> Can you teach data structures on top of basic Python programming in 15 minutes?</p></li>
<li><p>Relating to the previous point, <strong>zero-based indexing.</strong> “Why is the last index not <img src="https://latex.codecogs.com/png.latex?x_%7B50%7D"> when I did <code>np.zeros(50)</code>?”</p></li>
<li><p><strong>Struggling to wrap their heads around the imperative <code>for</code> loops and nested <code>for</code> loops.</strong></p></li>
<li><p>Relating to the previous point, <strong>not even knowing how to start iterating <code>x[n]</code> imperatively.</strong> Even without the <code>for</code> loop.</p></li>
</ol>
<p>There are of course other minor things like doing <code>Ax[n](1 - x[n])</code> instead of <code>A * x[n] * (1 - x[n])</code>, secretly copy-pasting from ChatGPT, not reading error messages, or getting <a href="../posts/tools-academic.html#marimo-over-jupyter">bamboozled by Jupyter’s hidden states</a>.</p>
<p>But ultimately, in my opinion, it mostly comes down to two things:</p>
<ul>
<li><strong><em>Syntax</em></strong> or little quirks of the language (Points 1 to 5).</li>
<li>Wrapping their heads around the <strong><em>imperative way</em></strong> of looping through each index and iterating each <img src="https://latex.codecogs.com/png.latex?x_n"> one by one (Points 6 and 7).</li>
</ul>
<div class="callout callout-style-default callout-warning callout-titled">
<div class="callout-header d-flex align-content-center">
<div class="callout-icon-container">
<i class="callout-icon"></i>
</div>
<div class="callout-title-container flex-fill">
<span class="screen-reader-only">Warning</span>Disclaimer
</div>
</div>
<div class="callout-body-container callout-body">
<p>I know I’m overstating this, as no one will actually be learning programming in 15 minutes. The goal is for the students to have fun interacting with Python along with cool visualizations that relates to physics, not to actually learn programming. Hence the 15 minutes “lesson” and 1.5 hours “hands-on”.</p>
<p>But humor me for a second, and consider what if it’s actually possible to learn <em>something</em> related to programming in 15 minutes?</p>
</div>
</div>
</section>
<section id="so-what-should-the-students-actually-be-learning" class="level1 page-columns page-full">
<h1>So what should the students actually be learning?</h1>
<p>Here’s my thesis:</p>
<p>Students should learn</p>
<ol type="1">
<li>the idea <strong><em>behind</em></strong> programming, not <strong><em>the</em></strong> programming,</li>
<li>to <strong>compose small behaviors</strong> into large solutions.</li>
</ol>
<p>You might be scratching your head here as this is quite vague, but these two points are really in the spirit of <a href="https://en.wikipedia.org/wiki/Edsger_W._Dijkstra">Edsger Dijkstra</a>’s well-known quote of</p>
<blockquote class="blockquote">
<p><em>“Computer science is no more about computers than astronomy is about telescopes.”</em></p>
</blockquote>
<section id="the-idea-behind-programming" class="level2 page-columns page-full">
<h2 class="anchored" data-anchor-id="the-idea-behind-programming">The idea behind programming</h2>
<p>For the first point, students shouldn’t be spending their time overcoming and “learning” the syntax and little quirks about the language.</p>
<p>If they are, and if they leave the class feeling like they learned something because Python eventually stopped throwing error messages at them, then we have really failed them in their education.</p>
<div class="callout callout-style-default callout-note callout-titled">
<div class="callout-header d-flex align-content-center">
<div class="callout-icon-container">
<i class="callout-icon"></i>
</div>
<div class="callout-title-container flex-fill">
<span class="screen-reader-only">Note</span>Important aside
</div>
</div>
<div class="callout-body-container callout-body">
<p>I might be being overly dramatic, but <strong>I think the point I’m making here is related to a crisis in modern education</strong>, where this is just one example.</p>
<p>In <a href="https://www.youtube.com/watch?v=Ia55clAtdMs">his TEDx talk</a>, <a href="https://en.wikipedia.org/wiki/Simon_Peyton_Jones">Simon Peyton Jones</a> notes that in school, what we teach falls broadly into two categories: <strong>skills</strong> and <strong>disciplines</strong>.</p>
<p>Skills are useful and immediately applicable knowledge, but becomes outdated quickly. Teaching Python, Microsoft Office, or more recently, using LLMs, are some examples.</p>
<p>Disciplines, by contrast, are foundational knowledge that makes us better than those who came before us. They are not as immediately applicable, but do not become outdated quickly; this is long-term knowledge and human wisdom. Subjects like Mathematics, Physics, History, and of course Computer Science etc. are some examples.</p>
<p>Simon observes that in his field of computer science, people have lost sight of the foundational discipline, and that there is too much focus on <strong>technology</strong> rather than <strong>ideas</strong>.</p>
<p>Perhaps this warrants a separate post, but not only do I think that Simon’s observations hold true for our entire learning culture, I think we are moving more and more towards this direction. We judge whether something is worth learning based on its “usefulness” in the current job market, and the result is a population that is highly educated on paper but not “in vivo” so to speak.</p>
</div>
</div>
<p>In the linked TEDx talk in the aside, Simon gave a video example of how some school children performed a sorting algorithm just by following some rules themselves, without any computers, bringing to life Dijkstra’s quote above, where “this clearly is about <em>computation</em> and not about technology”.</p>
<p>I wish we could demonstrate this for our class, but alas the students ended up debugging Python codes most of the time, and “learned” Python instead of something deeper.</p>
<p>So how can we do better?</p>
<p>I don’t really know, maybe we can go back to pre-2000s and teach <a href="https://en.wikipedia.org/wiki/Lisp_(programming_language)">Lisp</a> as it is typically praised for having lesser syntax and grammars than other languages<sup>1</sup>?</p>
<div class="no-row-height column-margin column-container"><div id="fn1"><p><sup>1</sup>&nbsp;Many universities, and most famously MIT until 2008, used to teach Lisp (or its dialects) for their computer science courses before Python and Java took over.</p></div><div id="fn2"><p><sup>2</sup>&nbsp;Working through SICM really is an example of Alan Perlis’s quote above.</p></div></div><p>Lisp still has some interests in the field of Physics, e.g., <a href="https://www.rigetti.com/">Rigetti</a>’s <a href="https://coalton-lang.github.io/20220906-quantum-compiler/">Quilc</a>, <a href="https://samritchie.io/">Sam Ritchie</a>’s <a href="https://www.youtube.com/watch?v=Jv2JgzAl5yU">Emmy</a>, and of course the popular <a href="https://en.wikipedia.org/wiki/Structure_and_Interpretation_of_Classical_Mechanics">SICM</a><sup>2</sup>. <a href="https://en.wikipedia.org/wiki/Robert_C._Martin">Uncle Bob</a> also <a href="https://blog.cleancoder.com/uncle-bob/2019/08/22/WhyClojure.html">thinks that</a> Lisp will outlasts all other languages and eventually becomes the standard language that all programmers use.</p>
</section>
<section id="learning-small-behaviors-and-composing-them-to-solve-big-problems" class="level2 page-columns page-full">
<h2 class="anchored" data-anchor-id="learning-small-behaviors-and-composing-them-to-solve-big-problems">Learning small behaviors and composing them to solve big problems</h2>
<p>To convince you of the second point, take a look at how you can do the first task in Haskell</p>

<div class="no-row-height column-margin column-container"><div class="margin-aside">
<p>Yes, this post turns out to be another thinly-veiled propaganda for functional programming.</p>
</div></div><div class="tabset-margin-container"></div><div class="panel-tabset">
<ul class="nav nav-tabs"><li class="nav-item"><a class="nav-link active" id="tabset-1-1-tab" data-bs-toggle="tab" data-bs-target="#tabset-1-1" aria-controls="tabset-1-1" aria-selected="true" href="">Haskell</a></li><li class="nav-item"><a class="nav-link" id="tabset-1-2-tab" data-bs-toggle="tab" data-bs-target="#tabset-1-2" aria-controls="tabset-1-2" aria-selected="false" href="">Python</a></li></ul>
<div class="tab-content">
<div id="tabset-1-1" class="tab-pane active" aria-labelledby="tabset-1-1-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb3" style="background: #f1f3f5;"><pre class="sourceCode haskell code-with-copy"><code class="sourceCode haskell"><span id="cb3-1">logisticMap a xn <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> a <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> xn <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> (<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span> xn)</span>
<span id="cb3-2"></span>
<span id="cb3-3">xs <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">take</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">50</span> (<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">iterate</span> (logisticMap <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0.5</span>) <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0.2</span>)</span>
<span id="cb3-4"></span>
<span id="cb3-5">main <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> plotList [] xs</span></code></pre></div></div>
</div>
<div id="tabset-1-2" class="tab-pane" aria-labelledby="tabset-1-2-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb4" style="background: #f1f3f5;"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb4-1">x <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> np.zeros(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">50</span>)</span>
<span id="cb4-2">x[<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0.2</span></span>
<span id="cb4-3"></span>
<span id="cb4-4">A <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0.5</span></span>
<span id="cb4-5"><span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">for</span> n <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">in</span> <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">range</span>(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">49</span>):</span>
<span id="cb4-6">    x[n<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> A <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> x[n] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> (<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span> x[n])</span>
<span id="cb4-7"></span>
<span id="cb4-8">plt.plot(x)</span></code></pre></div></div>
</div>
</div>
</div>
<p>and the second task</p>
<div class="tabset-margin-container"></div><div class="panel-tabset">
<ul class="nav nav-tabs"><li class="nav-item"><a class="nav-link active" id="tabset-2-1-tab" data-bs-toggle="tab" data-bs-target="#tabset-2-1" aria-controls="tabset-2-1" aria-selected="true" href="">Haskell</a></li><li class="nav-item"><a class="nav-link" id="tabset-2-2-tab" data-bs-toggle="tab" data-bs-target="#tabset-2-2" aria-controls="tabset-2-2" aria-selected="false" href="">Haskell (with plotting)</a></li><li class="nav-item"><a class="nav-link" id="tabset-2-3-tab" data-bs-toggle="tab" data-bs-target="#tabset-2-3" aria-controls="tabset-2-3" aria-selected="false" href="">Python</a></li></ul>
<div class="tab-content">
<div id="tabset-2-1" class="tab-pane active" aria-labelledby="tabset-2-1-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb5" style="background: #f1f3f5;"><pre class="sourceCode haskell code-with-copy"><code class="sourceCode haskell"><span id="cb5-1">task1 a <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">drop</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">300</span> (<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">take</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">600</span> (<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">iterate</span> (logisticMap a) <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0.2</span>))</span>
<span id="cb5-2"></span>
<span id="cb5-3">aValues <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> toList (linspace <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">3000</span> (<span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">3.4</span>, <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">4.0</span>))</span>
<span id="cb5-4"></span>
<span id="cb5-5">xs <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">map</span> task1 aValues</span></code></pre></div></div>
</div>
<div id="tabset-2-2" class="tab-pane" aria-labelledby="tabset-2-2-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb6" style="background: #f1f3f5;"><pre class="sourceCode haskell code-with-copy"><code class="sourceCode haskell"><span id="cb6-1">task1 a <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">drop</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">300</span> (<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">take</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">600</span> (<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">iterate</span> (logisticMap a) <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0.2</span>))</span>
<span id="cb6-2"></span>
<span id="cb6-3">aValues <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> toList (linspace <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">3000</span> (<span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">3.4</span>, <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">4.0</span>))</span>
<span id="cb6-4"></span>
<span id="cb6-5">points <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> [(a, xs) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|</span> a <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> aValues, xs <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> task1 a]</span>
<span id="cb6-6"></span>
<span id="cb6-7">main <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> plotPath [<span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">PlotStyle</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Points</span>] points</span></code></pre></div></div>
</div>
<div id="tabset-2-3" class="tab-pane" aria-labelledby="tabset-2-3-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb7" style="background: #f1f3f5;"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb7-1"><span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">for</span> A <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">in</span> np.linspace(<span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">3.4</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">4</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">3000</span>):</span>
<span id="cb7-2">    x <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> np.zeros(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">600</span>)</span>
<span id="cb7-3">    x[<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0.2</span></span>
<span id="cb7-4"></span>
<span id="cb7-5">    <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">for</span> n <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">in</span> <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">range</span>(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">599</span>):</span>
<span id="cb7-6">        x[n<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> A <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> x[n] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> (<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span> x[n])</span>
<span id="cb7-7"></span>
<span id="cb7-8">    plt.plot([A] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">300</span>, x[<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">300</span>:], <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">','</span>)</span></code></pre></div></div>
</div>
</div>
</div>

<div class="no-row-height column-margin column-container"><div class="margin-aside">
<p>I cheated a little here as I removed the type declarations and library imports for both Haskell solutions. Though, they are more related to the first point about quirks of the language than the second point that I’m trying to make here.</p>
<p>I’ve also separated the solution for plotting as again it involves quirks about the language, particularly list comprehension. Though, it simply corresponds to the mathematical form of <img src="https://latex.codecogs.com/png.latex?%5C%7B(A,%20x_n)%20%7C%20A%20%5Cin%20%5B3.4,%204%5D,%20x_n%20%5Cin%20%5Ctextrm%7Btask1%7D(A)%5C%7D">.</p>
</div></div><p>Of course, if one is unfamiliar with functional programming, one would be confused on what are <code>map</code>, <code>iterate</code>, <code>take</code>, and <code>drop</code>.</p>
<p>But turns out, these four functions appear in almost <strong>all functional programming languages</strong><sup>3</sup>. This way, learning programming stops being about learning syntax and becomes about learning vocabularies.</p>
<div class="no-row-height column-margin column-container"><div id="fn3"><p><sup>3</sup>&nbsp;Type of one them into <a href="https://hoogletranslate.com/">Hoogle Translate</a> and “translate” it to other languages.</p></div></div><p>We can teach the simple behaviors of these four functions (and more!) to the students in 15 minutes, and they will try to figure out how to put them together to solve complex problem.</p>
<p>Again, remember Dijkstra’s quote, or the school children performing a sorting algorithm without any computers; how to compose these functions together is independent of the programming language, and so we can simply use words to describe them, e.g.,</p>
<ol type="1">
<li>We <code>iterate</code> the logistic map function with some <img src="https://latex.codecogs.com/png.latex?A"> (ad infinitum), on an initial value of <img src="https://latex.codecogs.com/png.latex?0.2">.</li>
<li>We then <code>take</code> just the first 600 values, and <code>drop</code> the first 300 of these 600 values to get values from years 301 to 600.</li>
<li>We then <code>map</code> steps 1 and 2 to a range of values of <img src="https://latex.codecogs.com/png.latex?A%20%5Cin%20%5B3.4,%204%5D">.</li>
</ol>
<p>This is not very different from composing universal logic gates such as NAND, NOT, XOR, etc. to build complicated machines<sup>4</sup>, a feat that kids and teens are capable of as evident in games like Minecraft with its <a href="https://minecraft.wiki/w/Redstone_mechanics">redstone circuits</a>.</p>
<div class="no-row-height column-margin column-container"><div id="fn4"><p><sup>4</sup>&nbsp;Take a look at <a href="https://www.youtube.com/watch?v=7JK6qtpKLoQ">this video by peppidesu</a> on how functional programming is about composing smaller building blocks into more complex machinery.</p></div></div><p>Compare this with Python’s imperative ways of doing things:</p>
<ol type="1">
<li>We initialize an array <code>x</code> with 600 values.</li>
<li>We set the first index of the array <code>x[0]</code> to be <img src="https://latex.codecogs.com/png.latex?0.2">.</li>
<li>We loop through each index <code>x[n]</code> from 0 to 598, and compute the value for the next index <code>x[n+1]</code> for some <img src="https://latex.codecogs.com/png.latex?A">.</li>
<li>We keep only the last 300 values of <code>x</code> with <code>x[300:]</code>.</li>
<li>We then put steps 1 to 4 in a loop that loops over <img src="https://latex.codecogs.com/png.latex?A%20%5Cin%20%5B3.4,%204%5D">.</li>
</ol>
<p>In my opinion, in the imperative case, we are forcing our thinking and ideas to be compatible with the design of the language. For example, this solution only works because Numpy is an array programming language, and we have to work around quirks of its designs, e.g., the range of index to loop over is somewhat unintuitively 0 to 598. For languages that don’t have array data structures, or have different quirks about how arrays are used, our solution would change.</p>
<p>In contrast, in the functional case, it’s simply function applications, i.e., <code>map(drop(take(iterate(...))))</code>, or in pipe notation, <code>iterate(logisticMap, 0.2) |&gt; take(600) |&gt; drop(300) |&gt; map(aValues)</code><sup>5</sup>.</p>
<div class="no-row-height column-margin column-container"><div id="fn5"><p><sup>5</sup>&nbsp;I’m simplifying here, but the point still stands.</p></div></div><p>For example, if one is familiar with functional programming vocabularies, one can do the same for different languages:</p>
<div class="tabset-margin-container"></div><div class="panel-tabset">
<ul class="nav nav-tabs"><li class="nav-item"><a class="nav-link active" id="tabset-3-1-tab" data-bs-toggle="tab" data-bs-target="#tabset-3-1" aria-controls="tabset-3-1" aria-selected="true" href="">Julia</a></li><li class="nav-item"><a class="nav-link" id="tabset-3-2-tab" data-bs-toggle="tab" data-bs-target="#tabset-3-2" aria-controls="tabset-3-2" aria-selected="false" href="">R</a></li><li class="nav-item"><a class="nav-link" id="tabset-3-3-tab" data-bs-toggle="tab" data-bs-target="#tabset-3-3" aria-controls="tabset-3-3" aria-selected="false" href="">Rust</a></li><li class="nav-item"><a class="nav-link" id="tabset-3-4-tab" data-bs-toggle="tab" data-bs-target="#tabset-3-4" aria-controls="tabset-3-4" aria-selected="false" href="">Scala</a></li></ul>
<div class="tab-content">
<div id="tabset-3-1" class="tab-pane active" aria-labelledby="tabset-3-1-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb8" style="background: #f1f3f5;"><pre class="sourceCode julia code-with-copy"><code class="sourceCode julia"><span id="cb8-1"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">using</span> <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">IterTools</span></span>
<span id="cb8-2"></span>
<span id="cb8-3"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">logistic_map</span>(A) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> xn <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-&gt;</span> A <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> xn <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> (<span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span> xn)</span>
<span id="cb8-4"></span>
<span id="cb8-5"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">task1</span>(A) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span></span>
<span id="cb8-6">    <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0.2</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|&gt;</span></span>
<span id="cb8-7">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">iterated</span>(<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">logistic_map</span>(A)) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|&gt;</span></span>
<span id="cb8-8">    (x <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-&gt;</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">take</span>(x, <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">600</span>)) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|&gt;</span></span>
<span id="cb8-9">    (x <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-&gt;</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">drop</span>(x, <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">300</span>)) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|&gt;</span></span>
<span id="cb8-10">    collect</span>
<span id="cb8-11"></span>
<span id="cb8-12">xs <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">map</span>(task1, <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">range</span>(<span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">3.4</span>, <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">4</span>, length<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">3000</span>))</span></code></pre></div></div>
</div>
<div id="tabset-3-2" class="tab-pane" aria-labelledby="tabset-3-2-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb9" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb9-1"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">library</span>(purrr)</span>
<span id="cb9-2"></span>
<span id="cb9-3">logistic_map <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">function</span>(A) {</span>
<span id="cb9-4">  <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">function</span>(x) A <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> x <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> (<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span> <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span> x)</span>
<span id="cb9-5">}</span>
<span id="cb9-6"></span>
<span id="cb9-7">task1 <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">function</span>(A) {</span>
<span id="cb9-8">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">accumulate</span>(</span>
<span id="cb9-9">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">seq_len</span>(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">599</span>),</span>
<span id="cb9-10">    <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">.init =</span> <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0.2</span>,</span>
<span id="cb9-11">    <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">.f =</span> <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">function</span>(x, unused) <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">logistic_map</span>(A)(x)</span>
<span id="cb9-12">  ) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|&gt;</span></span>
<span id="cb9-13">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">tail</span>(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">300</span>)</span>
<span id="cb9-14">}</span>
<span id="cb9-15"></span>
<span id="cb9-16">xs <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">map</span>(</span>
<span id="cb9-17">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">seq</span>(<span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">3.4</span>, <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">4.0</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">length.out =</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">3000</span>),</span>
<span id="cb9-18">  task1</span>
<span id="cb9-19">)</span></code></pre></div></div>
</div>
<div id="tabset-3-3" class="tab-pane" aria-labelledby="tabset-3-3-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb10" style="background: #f1f3f5;"><pre class="sourceCode rust code-with-copy"><code class="sourceCode rust"><span id="cb10-1"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">fn</span> task1(a<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">f64</span>) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-&gt;</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Vec</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span><span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">f64</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb10-2">    <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">std::iter::</span>successors(<span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">Some</span>(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0.2</span>)<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">move</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|&amp;</span>x<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb10-3">        <span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">Some</span>(a <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> x <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> (<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1.0</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span> x))</span>
<span id="cb10-4">    <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span>)</span>
<span id="cb10-5">    <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>skip(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">300</span>)</span>
<span id="cb10-6">    <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>take(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">300</span>)</span>
<span id="cb10-7">    <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>collect()</span>
<span id="cb10-8"><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span>
<span id="cb10-9"></span>
<span id="cb10-10"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">fn</span> main() <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb10-11">    <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span> xs<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Vec</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span><span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Vec</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span><span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">f64</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;&gt;</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> linspace(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">3.4</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">4.0</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">3000</span>)</span>
<span id="cb10-12">        <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>into_iter()</span>
<span id="cb10-13">        <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>map(task1)</span>
<span id="cb10-14">        <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>collect()<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb10-15"><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span></code></pre></div></div>
</div>
<div id="tabset-3-4" class="tab-pane" aria-labelledby="tabset-3-4-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb11" style="background: #f1f3f5;"><pre class="sourceCode scala code-with-copy"><code class="sourceCode scala"><span id="cb11-1"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">def</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">logisticMap</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>a<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="ex" style="color: null;
background-color: null;
font-style: inherit;">Double</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)(</span>x<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="ex" style="color: null;
background-color: null;
font-style: inherit;">Double</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">):</span> <span class="ex" style="color: null;
background-color: null;
font-style: inherit;">Double</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span></span>
<span id="cb11-2">  a <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> x <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">1.0</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span> x<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span id="cb11-3"></span>
<span id="cb11-4"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">def</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">task1</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>a<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="ex" style="color: null;
background-color: null;
font-style: inherit;">Double</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">):</span> <span class="ex" style="color: null;
background-color: null;
font-style: inherit;">Vector</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">[</span><span class="ex" style="color: null;
background-color: null;
font-style: inherit;">Double</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">]</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span></span>
<span id="cb11-5">  <span class="ex" style="color: null;
background-color: null;
font-style: inherit;">Iterator</span></span>
<span id="cb11-6">    <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">iterate</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0.2</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)(</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">logisticMap</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>a<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">))</span></span>
<span id="cb11-7">    <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">drop</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">300</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span id="cb11-8">    <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">take</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">300</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span id="cb11-9">    <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>toVector</span>
<span id="cb11-10"></span>
<span id="cb11-11"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">val</span> xs<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="ex" style="color: null;
background-color: null;
font-style: inherit;">Vector</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">[</span><span class="ex" style="color: null;
background-color: null;
font-style: inherit;">Vector</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">[</span><span class="ex" style="color: null;
background-color: null;
font-style: inherit;">Double</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">]]</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span></span>
<span id="cb11-12">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">linspace</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">3.4</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">4.0</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">3000</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">).</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">map</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>task1<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span></code></pre></div></div>
</div>
</div>
</div>

<div class="no-row-height column-margin column-container"><div class="margin-aside">
<p>Note that in the case of R, we used a different vocabulary, <code>accumulate</code>, instead, which is also known as <code>scan</code> in Haskell.</p>
<p>Also note that in the case of Rust and Scala, there is actually no <code>linspace</code> function, so you’ll have to roll your own or use a third-party package.</p>
</div></div><p>While different languages use different names for the same vocabularies, e.g., <code>iterated</code>, <code>tail</code>, <code>skip</code>, <code>successors</code> etc., in my opinion, such a functional way of doing things makes it easier to reason about the code, and much more difficult to make a logical error.</p>
</section>
</section>
<section id="does-it-actually-matter" class="level1">
<h1>Does it actually matter?</h1>
<p>Perhaps not, as the goal is for the students to have some fun playing around with programming. In fact, a number of students have told us that they enjoyed the class and felt a sense of accomplishment when Python finally plots the pretty plots after they navigated all the errors.</p>
<p>Furthermore, the entire industry is stuck on imperative programming anyway, and Python programming is widely used in Physics careers. So it is no surprise that we are forced to teach Python (and in Jupyter notebooks) as part of the University’s policy.</p>
<p>Still, it doesn’t hurt to dream a little of what might be possible in teaching programming in a manner that is deeper, more transferable, and engages the students’ thinking more than just learning the programming language or technology, especially considering the rise of AI agents that can <a href="https://en.wikipedia.org/wiki/GPT-6_Astra">program an entire 3D game in a prompt</a>.</p>
<p>Finally, I’ll end with this quote</p>
<blockquote class="blockquote">
<p><em>“When the limestone of imperative programming has worn away, the granite of functional programming will be revealed underneath.”</em></p>
<div style="text-align: right;">
<p>— Simon Peyton Jones</p>
</div>
</blockquote>
<p>Until then, I suppose we’ll keep feeding this vicious cycle of teaching students imperative Python and Jupyter notebooks, even when they will be outsourcing it to ChatGPT anyway.</p>


</section>


<a onclick="window.scrollTo(0, 0); return false;" id="quarto-back-to-top"><i class="bi bi-arrow-up"></i> Back to top</a> ]]></description>
  <category>programming</category>
  <guid>https://jianweicheong.codeberg.page/posts/program-15-min.html</guid>
  <pubDate>Sat, 05 Sep 2026 16:00:00 GMT</pubDate>
</item>
<item>
  <title>Spooky action at a distance is NOT about entanglement</title>
  <link>https://jianweicheong.codeberg.page/posts/spooky-action.html</link>
  <description><![CDATA[ 




<p>If you look up the phrase “<em>spooky action at a distance</em>”, be it on YouTube, Google, or an LLM, you’ll most likely get the answer that it refers to the phenomenon of <em><a href="https://en.wikipedia.org/wiki/Quantum_entanglement">quantum entanglement</a></em>.</p>
<p>There are myriads of YouTube videos or Reddit comments, where when explaining “spooky action at a distance” to the general public, people default to the standard pop-sci explanation that concerns entanglement.</p>
<p>It goes something like this:</p>
<blockquote class="blockquote">
<ol type="1">
<li>Two quantum particles are somehow made to be correlated with each other, e.g., if one is <em>spin up</em>, the other must be <em>spin down</em>.</li>
<li>Emphasize that but each of them are neither <em>spin up</em> nor <em>spin down</em> before “observation”, and are in a “superposition” instead.</li>
<li>The two particles are then separated by a great distance.</li>
<li>Next, bewilder your audience by claiming that when you “observe” one particle, and say it turns out to be <em>spin up</em>, the other particle must instantly becomes <em>spin down</em>, despite the great distance.</li>
<li>Finally, claim that this is because the two particles are entangled, and that this entanglement connects the two particles across the great distance.</li>
</ol>
</blockquote>
<p>For the learned reader however, these five steps translate to:</p>
<ol type="1">
<li>Two particles are entangled,</li>
<li>and entangled particles are in a superposition. <img src="https://latex.codecogs.com/png.latex?%0A%5Cfrac%7B1%7D%7B%5Csqrt%7B2%7D%7D(%7C%5Cuparrow%5Cdownarrow%5Crangle%20+%20%7C%5Cdownarrow%5Cuparrow%5Crangle)%0A"></li>
<li>Measurement leads to the <strong><em>instantaneous collapse of the wavefunction</em></strong> to the measurement basis (typically the basis states that are in superposition). <img src="https://latex.codecogs.com/png.latex?%0A%7C%5Cuparrow%5Cdownarrow%5Crangle%20%5Cquad%20%5Ctext%7Bor%7D%20%5Cquad%20%7C%5Cdownarrow%5Cuparrow%5Crangle%0A"></li>
</ol>
<p>Wait a minute… did I just say “<em>instantaneous collapse of the wavefunction</em>”?</p>
<p>As in the phenomenon that physics undergraduates learn that applies to all wavefunctions during a measurement, <strong>with or without</strong> entanglement? You know, the one where you have to apply the <em><a href="https://en.wikipedia.org/wiki/Born_rule">Born rule</a></em> suddenly, and to update the wavefunction by hand manually instead of using the <em><a href="https://en.wikipedia.org/wiki/Schr%C3%B6dinger_equation">Schrödinger equation</a></em> without understanding why? The one that is a <strong>separate phenomenon from entanglement</strong>? <img src="https://latex.codecogs.com/png.latex?%0A%5Cfrac%7B1%7D%7B%5Csqrt%7B2%7D%7D%7C%5Cuparrow%5Crangle%20+%20%7C%5Cdownarrow%5Crangle%20%5Cquad%20%5Clongrightarrow%20%5Cquad%20%7C%5Cuparrow%5Crangle%20%5Cquad%20%5Ctext%7Bor%7D%20%5Cquad%20%7C%5Cdownarrow%5Crangle%0A"></p>
<p>In fact, this pop-sci description always consists of two parts: the entanglement of the two particles, followed by the measurement of one particle. Quite plainly, the magic happens in the second step, the measurement process, and does not happen in the first step with just entanglement by itself.</p>
<p>So what is happening? Why do so many people get it wrong<sup>1</sup>?</p>
<div class="no-row-height column-margin column-container"><div id="fn1"><p><sup>1</sup>&nbsp;In the “<a href="https://en.wikipedia.org/wiki/Educational_entertainment">edutainment</a>” space, I’ve only came across two instances where this was explained correctly: <a href="https://www.youtube.com/@veritasium">Veritasium</a> and <a href="https://en.wikipedia.org/wiki/Sabine_Hossenfelder">Sabine Hossenfelder</a>.</p></div></div><p>You see, the phrase “<em>spooky action at a distance</em>” is a pop-sci term, and rarely or never comes up in the standard textbooks of quantum mechanics. As such, the standard pop-sci description of it has dominated and spread like a virus into the minds of physicists and physics-enthusiasts alike. We would just default to parrotting the same things without thinking too much into it.</p>
<p>But any physics undergraduate who paused for a moment, and thought about it for awhile, should have realized that it refers to the deeper problem/phenomenon of a measurement rather than entanglement.</p>
<p>And it is a shame that this deeper problem is often neglected or conflated with entanglement despite its deep and philosophical implications that concerns free will and the nature of reality.</p>
<section id="but-what-about-einsteins-epr-paper" class="level1 page-columns page-full">
<h1>But what about Einstein’s EPR paper?</h1>
<p>Let’s look at the timeline:</p>
<ul>
<li><p>1927: Einstein first raised his issues with quantum mechanics at the fifth Solvey conference, arguing that it is incomplete. He used the term “peculiar mechanism of action at a distance” <span class="citation" data-cites="BacciagaluppiQuantum2009">(Bacciagaluppi and Valentini 2009, 485–87)</span>.</p></li>
<li><p>1935: Einstein, Podolsky, and Rosen published the famous <a href="https://en.wikipedia.org/wiki/Einstein%E2%80%93Podolsky%E2%80%93Rosen_paradox">EPR paper</a> that introduced quantum entanglement. There is no mention of the phrase “action at a distance” in the paper <span class="citation" data-cites="EinsteinEPR1935">(Einstein, Podolsky, and Rosen 1935)</span>.</p></li>
<li><p>1947: In a letter to Max Born where Einstein maintains his position that quantum mechanics is incomplete, the term “spooky actions at a distance” was used and popularized <span class="citation" data-cites="BornBornEinstein1971">(Born et al. 1971, 157–58)</span>.</p></li>
</ul>
<p>It seems to me that the Einstein’s “action at a distance” problem with quantum mechanics predates the concept of quantum entanglement, and that the case of the two entangled particles in the EPR paper that everyone parrots now is simply one example that demonstrates the issue.</p>
<p>Furthermore, in the context of the letters to Max Born in 1947:</p>
<blockquote class="blockquote">
<p>“I cannot seriously believe in it because the theory cannot be reconciled with the idea that physics should represent a reality in time and space, free from spooky actions at a distance.”</p>
</blockquote>
<blockquote class="blockquote">
<p>“But I am quite convinced that someone will eventually come up with a theory whose objects, connected by laws, are not probabilities but considered facts, as used to be taken for granted until quite recently.”</p>
</blockquote>

<div class="no-row-height column-margin column-container"><div class="margin-aside">
<p>On an unrelated note, it blows my mind how difficult it is to find an accessible copy of this document, despite it being 70 years old and so well-known.</p>
</div></div><p>It seems clear to me that Einstein’s problems with “spooky actions at a distance” concerns the nature of “realism” and it’s contradiction with “locality”, rather than entanglement.</p>
<p>This should be clear if we look at his 1927 example at the fifth Solvey conference.</p>
<section id="einsteins-1927-example-without-entanglement" class="level2 page-columns page-full">
<h2 class="anchored" data-anchor-id="einsteins-1927-example-without-entanglement">Einstein’s 1927 example without entanglement</h2>
<div class="quarto-figure quarto-figure-center">
<figure class="figure">
<p><img src="https://jianweicheong.codeberg.page/media/posts/spooky-action/einstein1927.png" class="img-fluid figure-img"></p>
<figcaption>Reproduced from <span class="citation" data-cites="BacciagaluppiQuantum2009">Bacciagaluppi and Valentini (2009)</span>, pp.&nbsp;486.</figcaption>
</figure>
</div>
<p>I’ll paraphrase a little:</p>
<blockquote class="blockquote">
<p>Consider sending a single particle through a single slit that lands on the (hemisphere) screen. The particle will diffract through the slit, with a position wavefunction <img src="https://latex.codecogs.com/png.latex?%5CPsi(x)"> that spreads out due to the diffraction. However, when the particle lands on the screen and is detected at just a single position <img src="https://latex.codecogs.com/png.latex?x'">, quantum mechanics dictates that the wavefunction <img src="https://latex.codecogs.com/png.latex?%5CPsi(x)"> will collapse to just a single position that is localized at <img src="https://latex.codecogs.com/png.latex?x'"> (a Dirac delta), with probability <img src="https://latex.codecogs.com/png.latex?%7C%5CPsi(x')%7C%5E2">.</p>
</blockquote>
<p>This is just standard textbook quantum mechanics.</p>
<blockquote class="blockquote">
<p>What this means however, is that when the particle is detected at position <img src="https://latex.codecogs.com/png.latex?x'"> on the screen, there is a “peculiar mechanism of action at a distance” that instantaneously prevents the particle from being detected at any other point on the screen (i.e., collapsing the wavefunction), regardless of its distance from <img src="https://latex.codecogs.com/png.latex?x'">.</p>
</blockquote>
<p>One might imagine that the left and the right ends of the (hemisphere) screen are a great distance from each other. Right before the particle’s wavefunction hits the screen, the wavefunction exists on both the left and the right ends. But the instant it hits the screen, say at the right end, instantaneously, the wavefunction that existed at the left end of the screen must disappears under some “peculiar mechanism of action at a distance”, even though the left end of the screen is causally disconnected from what happens at the right end of the screen.</p>
<p>Einstein argues that if we take the viewpoint that the wavefunction <img src="https://latex.codecogs.com/png.latex?%5CPsi(x)"> is indeed a complete description of the particle’s position, then we must consider that the particle is potentially present at all <img src="https://latex.codecogs.com/png.latex?x"> where <img src="https://latex.codecogs.com/png.latex?%7C%5CPsi(x)%7C%5E2"> is nonzero (which can span a great distance), and give up on locality when it ends up appearing at just a single <img src="https://latex.codecogs.com/png.latex?x'">.</p>
<p>Einstein instead prefers the viewpoint that the wavefunction <img src="https://latex.codecogs.com/png.latex?%5CPsi(x)"> represents just a probabilistic ensemble due to our lack of knowledge (of hidden variables), and that in reality the particle has just a single <em>actual</em> path. In other words, the wavefunction <img src="https://latex.codecogs.com/png.latex?%5CPsi(x)"> (and thus quantum mechanics) is not a complete description of the single particle.</p>

<div class="no-row-height column-margin column-container"><div class="margin-aside">
<p>These two different viewpoints where we either give up locality or the ontic existence of the path of the particle is related to the discussions regarding “<a href="https://en.wikipedia.org/wiki/Bell's_theorem">local realism</a>”.</p>
</div></div><p>This example touches on the main issue of “spooky action at a distance” (but without any mention of entanglement), where the wavefunction is “updated” instantaneously regardless of distances when it is measured at just one location.</p>
</section>
</section>
<section id="hardys-modern-example" class="level1 page-columns page-full">
<h1>Hardy’s modern example</h1>
<p>We have come a long way since then, and there are now many examples that demonstrate this intuitively and formally without the long-winded explanation about entanglement.</p>
<p>Here’s an example provided by <a href="https://en.wikipedia.org/wiki/Lucien_Hardy">Lucien Hardy</a> <span class="citation" data-cites="HardyEPR1995">(Hardy 1995)</span><sup>2</sup>.</p>
<div class="no-row-height column-margin column-container"><div id="fn2"><p><sup>2</sup>&nbsp;I cheated a little here, as to formalize this example mathematically, the concept of entanglement is still required. The detection of a photon in one path is entangled to a “vacuum mode” at the other path, i.e., a non-detection. But I’ll argue that in terms of the intuitive understanding of the setup, no mention of entanglement is necessary.</p></div></div><div id="fig-hardy" class="quarto-layout-panel">
<figure class="quarto-float quarto-float-fig figure">
<div aria-describedby="fig-hardy-caption-0ceaefa1-69ba-4598-a22c-09a6ac19f8ca">
<div class="quarto-layout-row">
<div class="quarto-layout-cell" style="flex-basis: 50.0%;justify-content: flex-start;">
<div class="quarto-figure quarto-figure-center">
<figure class="figure">
<p><img src="https://jianweicheong.codeberg.page/media/posts/spooky-action/hardy.svg" class="img-fluid figure-img"></p>
<figcaption>Before measurement.</figcaption>
</figure>
</div>
</div>
<div class="quarto-layout-cell" style="flex-basis: 50.0%;justify-content: flex-start;">
<div class="quarto-figure quarto-figure-center">
<figure class="figure">
<p><img src="https://jianweicheong.codeberg.page/media/posts/spooky-action/hardy2.svg" class="img-fluid figure-img"></p>
<figcaption>After measurement.</figcaption>
</figure>
</div>
</div>
</div>
</div>
<figcaption class="quarto-float-caption-bottom quarto-float-caption quarto-float-fig" id="fig-hardy-caption-0ceaefa1-69ba-4598-a22c-09a6ac19f8ca">
Figure&nbsp;1: The red lines visualize the position wavefunction of the photon, where the hump represents the probability of finding the photon. Before measurement, it spans both paths, but once the detector measured the photon to be found in path 1, it must collapses at path 2 even if it is a great distance away.
</figcaption>
</figure>
</div>
<blockquote class="blockquote">
<ol type="1">
<li>Send a single photon to a beam splitter.</li>
<li>The beam splitter has a 50% chance of letting the photon pass through, and a 50% chance of deflecting the photon to another path.</li>
<li>The single photon is now in a superposition of two paths, and we can let the photon propagate as far as we want down the two paths, so that the two paths are separated by a great distance.</li>
<li>Supposed we measure the photon in path 1 using a detector, and found it to be in path 1.</li>
<li>Quantum mechanics dictates that instantaneously, the “potential” photon in path 2 must disappears, regardless of the great distance between the two paths, such that a detector at path 2 can now never detect the photon at path 2.</li>
</ol>
</blockquote>
<p>Additionally, if instead we were to remove the detector at the very last minute before the photon would have hit the detector in path 1, then a detector at path 2 can still potentially detect the photon at path 2, as though like our choice of putting the detector in path 1 affects what happens in path 2 a great distance away.</p>
<p>Hardy took Redhead’s definition of locality of <span class="citation" data-cites="RedheadIncompleteness1987">(Redhead 1987)</span>: “elements of physical reality pertaining to one system cannot be affected by measurements performed ‘at-a-distance’ on another system.”</p>
<p>If we assume that the photon’s paths down either path 1 or path 2 are “ontologically real”, then our action of putting a detector at path 1 somehow affects the “reality” in path 2, regardless of the great distance separating them, i.e., we have to give up on locality.</p>
<section id="a-more-cartoonish-example-with-schrödingers-cat" class="level2">
<h2 class="anchored" data-anchor-id="a-more-cartoonish-example-with-schrödingers-cat">A more cartoonish example with Schrödinger’s cat</h2>
<p>Since the <a href="https://en.wikipedia.org/wiki/Schr%C3%B6dinger%27s_cat">Schrödinger’s cat thought experiment</a> is quite well-known among the general public, perhaps augmenting it to explain the underlying issue might be the way to go.</p>
<p>The original Schrödinger’s cat thought experiment goes like this:</p>
<blockquote class="blockquote">
<ol type="1">
<li>A cat is kept in a box that is isolated from the rest of the world.</li>
<li>In the box is a vial of poison gas that is rigged to be released to kill the cat depending on the measurement outcome of a quantum system that has a 50% chance of occurring (typically detecting a radioactive decay that has a 50% chance of decaying).</li>
<li>Because of this, before opening the box to the outside world, the cat is in a quantum superposition of being dead and alive.</li>
<li>But once the box is opened, the cat will collapse to a single state of either dead or alive.</li>
</ol>
</blockquote>
<p>Consider instead that if the vial of poison did not release, instead of simply leaving the cat alive, a trap door opens beneath the cat, dropping it into an underground tunnel that connects to your neighbor’s house.</p>
<p>The bored cat can then go through the long tunnel and enters your neighbor’s house, in which it will take a nap on the couch.</p>
<p>Now, before opening the box in your house to check for the cat, the cat is in a superposition of being dead in the box locally in your house, or alive and asleep some distance away in your neighbor’s house.</p>
<p>So if you were to open the box to check, quantum mechanics dictates that instantaneously, the action that you performed locally in your house, somehow affects what happens in your neighbor’s house despite that it is a distance away.</p>
<p>This is the “spooky action at a distance”.</p>
<p>The state of the cat spans a great distance between the two houses, but somehow changes instantaneously <em>everywhere</em> based on actions performed only locally.</p>
</section>
</section>
<section id="so-where-are-we-at-now" class="level1 page-columns page-full">
<h1>So where are we at now?</h1>
<p>One reason this <em><a href="https://en.wikipedia.org/wiki/Measurement_problem">measurement problem</a></em> is often ignored is because there is already a mainstream interpretation that physicists learn in textbooks<sup>3</sup>.</p>
<div class="no-row-height column-margin column-container"><div id="fn3"><p><sup>3</sup>&nbsp;Of course, there is a zoo of interpretations of quantum mechanics, the most well-known of which is the <em><a href="https://en.wikipedia.org/wiki/Many-worlds_interpretation">many-worlds interpretation</a></em>.</p></div></div><p>This is the regnant <a href="https://en.wikipedia.org/wiki/Copenhagen_interpretation">Copenhagen interpretation</a>, where it is accepted that there is indeed no “real” existence of the cat at either houses before measurement, and that we just have to accept the manual updating of the collapse of the wavefunction by hand.</p>
<p>To quote Wikipedia: “no truth can be attributed to an object except according to the results of its measurement”. In other words, it makes little sense to ask why a measurement at your house can affect the “reality” at your neighbor’s house, if we accept that there is no “truth” statement that can be made about a cat at your neighbor’s house before measurement in the first place.</p>
<p>Sometimes, this interpretation manifests as the students learning that the wavefunction is simply a “probability wave” describing the position of the particle, and has no real existence until measurement. This of course runs the risk of conflating with Einstein’s preferred view that it is only a probabilistic description because of our lack of information (of hidden variables)<sup>4</sup>, but I guess students are typically more concerned with being able to solve an exam problem rather than the philosophical aspects anyway.</p>
<div class="no-row-height column-margin column-container"><div id="fn4"><p><sup>4</sup>&nbsp;The real answer is that we just don’t know what the wavefunction is physically, or that it simply is its own definition without any analog for our intuition.</p></div></div></section>
<section id="appendix" class="level1 page-columns page-full">
<h1>Appendix</h1>
<section id="but-entanglement-is-still-a-connection-across-distances-right" class="level2 page-columns page-full">
<h2 class="anchored" data-anchor-id="but-entanglement-is-still-a-connection-across-distances-right">But entanglement is still a connection across distances right?</h2>
<p>Not really.</p>
<p>Entanglement is simply correlations, and correlations, once created, can be maintained over great distances, thus nonlocal, regardless if it is quantum or classical. Correlations are not connections.</p>
<p>Indeed, if we have two entangled particles, <img src="https://latex.codecogs.com/png.latex?A"> and <img src="https://latex.codecogs.com/png.latex?B">, and if we were to do some operation to <img src="https://latex.codecogs.com/png.latex?A">, the other particle <img src="https://latex.codecogs.com/png.latex?B"> will NOT be affected. What will happen instead is that the correlations/entanglement between them weaken as we just perturbed our particle <img src="https://latex.codecogs.com/png.latex?A"> away from its initial state that was much more correlated/entangled with <img src="https://latex.codecogs.com/png.latex?B">.</p>
<p>Formally, we call this a <em>local operation</em>. And local operations cannot increase correlations/entanglement, nor can it affect the other particle that we did not perform the operation on.</p>
<p>In fact, the intuition behind this should be quite straightforward and do not detract from our everyday lives.</p>
<p>For example, supposed if we print out two copies of a picture, and give each to Alice and Bob. Since the pictures are the same, they are correlated.</p>
<p>Now, Alice can draw something on her own copy of the picture, but doing so will inevitably make her picture different from Bob’s, i.e., correlations must decrease.</p>
<p>The only way to increase the correlation, is for Alice and Bob to communicate, and draw on their copies together in an agreed upon way, i.e., drawing the same thing<sup>5</sup>.</p>
<div class="no-row-height column-margin column-container"><div id="fn5"><p><sup>5</sup>&nbsp;Note that this method can only increase classical correlations, and not entanglement (quantum correlations). Formally, this is called <a href="https://en.wikipedia.org/wiki/LOCC"><em>local operation and classical communication</em> (LOCC)</a>.</p></div></div><p>There are no connections connecting the two pictures. They are simply correlated, and Alice and Bob can make them more or less correlated by performing operations on their own copies.</p>
<p>Instead, the “magic” behind all the things you heard about entanglement somehow being connected across distances IS the instantaneous collapse of the wavefunction, like what we have been discussing throughout this post.</p>
<p>It is the exception that is different from just “<em>local operation</em>” (see the next part).</p>
</section>
<section id="so-what-exactly-is-the-weirdness-about-quantum-mechanics" class="level2 page-columns page-full">
<h2 class="anchored" data-anchor-id="so-what-exactly-is-the-weirdness-about-quantum-mechanics">So what exactly is the weirdness about quantum mechanics?</h2>
<p>In my opinion, there are three main aspects to the measurement problem (that all relates to each other) that when put together, are the weirdest and deepest parts about quantum mechanics.</p>
<ol type="1">
<li>Detraction from continuous evolution (e.g., from Schrödinger equation)<sup>6</sup>.</li>
<li>Probabilistic nature (or true randomness).</li>
<li>Instantaneous and nonlocal influence of the wavefunction.</li>
</ol>
<div class="no-row-height column-margin column-container"><div id="fn6"><p><sup>6</sup>&nbsp;This is what makes it different from normal <em>operations</em> in the previous part.</p></div></div><p>For the most part above, we only touched upon the third aspect of the “spooky” instantaneous and nonlocal update of the wavefunction. But the first two aspects are perhaps more interesting.</p>
<p>Whenever a quantum system evolves, it can be described by the Schrödinger equation<sup>7</sup>. The quantum system evolves deterministically and in a continuous way. In other words, given the initial state of the quantum system, we can predict and compute the state at any future time.</p>
<div class="no-row-height column-margin column-container"><div id="fn7"><p><sup>7</sup>&nbsp;I’m oversimplifying as there’s also other kinds of evolution. But the point still stands.</p></div></div><p>Now this is not anything surprising. In fact, this is the case for <strong>ALL known laws of physics</strong>.</p>
<p>All, except a quantum measurement.</p>
<p>A quantum measurement is the only known physics of nature, out of the thousands if not tens of thousands of laws and behaviors that spans all of nature and human disciplines, that is not deterministic and features a discontinuous change from all past states<sup>8</sup>.</p>
<div class="no-row-height column-margin column-container"><div id="fn8"><p><sup>8</sup>&nbsp;There are of course seemingly random or discontinuous behaviors in many systems in many domains, but those are due to a lack of information about the full system (like how Einstein thinks is also the case for quantum mechanics). I am instead assuming a <a href="https://en.wikipedia.org/wiki/Laplace%27s_demon">Laplace’s demon</a> situation where even if all information are accessible, we still see such a discontinuous jump from past states.</p></div><div id="fn9"><p><sup>9</sup>&nbsp;Yes, even when you roll a dice or generate a random number on your computer <a href="https://en.wikipedia.org/wiki/Lavarand">from the fluctuations of lava lamps</a>, all these are only <a href="https://en.wikipedia.org/wiki/Pseudorandomness">pseudorandom</a>, and are actually deterministic. A small caveat (and still widely debated) is that if quantum fluctuations or uncertainties can survive and propagate to the macroscopic realm without averaging out in some <a href="https://en.wikipedia.org/wiki/Quantum_chaos">nonlinear chaotic dynamics</a>, then it might be possible for the macroscopic dynamics to also be random. You might also notice that either way, free will is inconsistent with the current known laws of physics, as it fits neither determinism nor randomness.</p></div></div><p>This discontinuous jump in the state of a quantum system (as far as mainstream opinion goes) is the only true randomness in all of known reality<sup>9</sup>.</p>
<p>For the computer scientists, this is as if the program of reality is written completely in <a href="https://en.wikipedia.org/wiki/Pure_function">pure deterministic functions</a> with no <a href="https://en.wikipedia.org/wiki/Side_effect_(computer_science)">side effects</a>. All functions, except a single one that somehow requires an <a href="https://en.wikipedia.org/wiki/Monad_(functional_programming)">IO monad</a>.</p>
<p>I find this to be the deepest aspect of quantum mechanics, and perhaps all of physics.</p>
<p>Some people, like Einstein (see the letter to Max Born above), thinks that reality shouldn’t be this way, and that <em>everything</em> should indeed be deterministic and must follows from past states. It seems that the only way forward for this view is to either exploit an unclosed loophole of <a href="https://en.wikipedia.org/wiki/Bell%27s_theorem">Bell’s theorem</a> to take the view of <a href="https://en.wikipedia.org/wiki/Superdeterminism">superdeterminism</a>, or to give up on locality. For most however, this random and discontinuous nature is simply accepted or ignored under the Copenhagen interpretation.</p>
<p>Personally, I think that there is a lot more behind the measurement problem, and I am optimistic that we will eventually uncover a deeper understanding of it one day.</p>



</section>
</section>


<a onclick="window.scrollTo(0, 0); return false;" id="quarto-back-to-top"><i class="bi bi-arrow-up"></i> Back to top</a><div id="quarto-appendix" class="default"><section class="quarto-appendix-contents" id="quarto-bibliography"><h2 class="anchored quarto-appendix-heading">References</h2><div id="refs" class="references csl-bib-body hanging-indent" data-entry-spacing="0">
<div id="ref-BacciagaluppiQuantum2009" class="csl-entry">
Bacciagaluppi, Guido, and Antony Valentini. 2009. <span>“Quantum Theory at the Crossroads: Reconsidering the 1927 Solvay Conference.”</span> <a href="https://arxiv.org/abs/quant-ph/0609184">https://arxiv.org/abs/quant-ph/0609184</a>.
</div>
<div id="ref-BornBornEinstein1971" class="csl-entry">
Born, Max, Hedwig Born, Irene Born, and Albert Einstein. 1971. <em>The Born-Einstein Letters: Correspondence Between Albert Einstein and Max and Hedwig Born from 1916 to 1955</em>. Walker.
</div>
<div id="ref-EinsteinEPR1935" class="csl-entry">
Einstein, A., B. Podolsky, and N. Rosen. 1935. <span>“Can Quantum-Mechanical Description of Physical Reality Be Considered Complete?”</span> <em>Physical Review</em> 47 (10): 777–80. <a href="https://doi.org/10.1103/physrev.47.777">https://doi.org/10.1103/physrev.47.777</a>.
</div>
<div id="ref-HardyEPR1995" class="csl-entry">
Hardy, Lucien. 1995. <span>“The EPR Argument and Nonlocality Without Inequalities for a Single Photona.”</span> <em>Annals of the New York Academy of Sciences</em> 755 (1): 600–615. <a href="https://doi.org/10.1111/j.1749-6632.1995.tb39000.x">https://doi.org/10.1111/j.1749-6632.1995.tb39000.x</a>.
</div>
<div id="ref-RedheadIncompleteness1987" class="csl-entry">
Redhead, Michael. 1987. <span>“Incompleteness, Non Locality and Realism. A Prolegomenon to the Philosophy of Quantum Mechanics.”</span> <em>Revue Philosophique de La France Et de l</em> 180 (4).
</div>
</div></section></div> ]]></description>
  <category>physics</category>
  <category>philosophy</category>
  <guid>https://jianweicheong.codeberg.page/posts/spooky-action.html</guid>
  <pubDate>Wed, 05 Aug 2026 16:00:00 GMT</pubDate>
</item>
<item>
  <title>R is still where the heart is</title>
  <link>https://jianweicheong.codeberg.page/posts/r-heart.html</link>
  <description><![CDATA[ 




<p>I first became interested in <a href="https://en.wikipedia.org/wiki/R_(programming_language)">R</a> after discovering <a href="https://rmarkdown.rstudio.com/">R Markdown</a> many years ago, but have only really used it properly in a graduate course dealing with bioinformatics.</p>
<p>Yet, despite spending orders of magnitude more time in Julia and Python for my work, I keep getting drawn back to R.</p>
<p>I just can’t get over the neat <a href="https://en.wikipedia.org/wiki/Tidyverse">tidyverse</a> way of data wrangling and the beautiful <a href="https://en.wikipedia.org/wiki/Wilkinson%27s_Grammar_of_Graphics">grammar of graphics</a> style of <a href="https://ggplot2.tidyverse.org/">ggplot2</a>.</p>
<p>Yes, there are packages in Julia and Python that replicates them, e.g., <a href="https://seaborn.pydata.org/">seaborn</a>, <a href="https://plotnine.org/">plotnine</a>, <a href="https://aog.makie.org/">AlgebraOfGraphics.jl</a>, <a href="https://tidierorg.github.io/Tidier.jl">Tidier.jl</a>, etc. But these are just further testaments to the awesomeness of R that other workflows in other languages are trying to replicate.</p>
<section id="some-examples" class="level1 page-columns page-full">
<h1>Some examples</h1>
<p>I should mention again, just like I did in the <a href="../posts/haskell-numerical.html">post on Haskell</a>, that I am neither a software developer nor am I computer science trained, and so some technical aspects of the language might escape me.</p>
<p>Additionally, I am not really an expert in all of the packages mentioned, and so there might be better solutions that I missed.</p>
<p>Having said that, let’s look at some examples for why I think R is awesome.</p>
<section id="clean-syntax" class="level2 page-columns page-full">
<h2 class="anchored" data-anchor-id="clean-syntax">Clean syntax</h2>
<p>In scientific computing, one typically has some function, say <code>hugeQuantumSys(x1, x2, x3, x4, ...)</code>, that takes in a bunch of parameters and return a whole other bunch of output values.<sup>1</sup>. Each of the input parameters has some range that we want to loop over to “scan” the entire parameter regime of the simulated physical system.</p>
<div class="no-row-height column-margin column-container"><div id="fn1"><p><sup>1</sup>&nbsp;Typically these input and output parameters correspond to some physical parameters of the system you are simulating, e.g., the initial state of the qubits, temperature of the interacting environment, entropy or energy of the qubits, etc.</p></div></div><p>After this huge and expensive computation, which ideally is parallelized over all input parameters, we most likely want to save all the input parameters with their corresponding output values in a file, say a <a href="https://en.wikipedia.org/wiki/Apache_Parquet">parquet file</a>.</p>
<p>What’s the best way to do this?</p>
<p>It’s obvious that we can use a dataframe, e.g., <a href="https://pandas.pydata.org/">pandas</a>, <a href="https://pola.rs/">polars</a>, <a href="https://dataframes.juliadata.org/">DataFrames.jl</a> etc., so that we can store all the input parameters and output values into a columnar format to be saved as a parquet file.</p>
<div class="tabset-margin-container"></div><div class="panel-tabset">
<ul class="nav nav-tabs"><li class="nav-item"><a class="nav-link active" id="tabset-1-1-tab" data-bs-toggle="tab" data-bs-target="#tabset-1-1" aria-controls="tabset-1-1" aria-selected="true" href="">R (tidyverse)</a></li><li class="nav-item"><a class="nav-link" id="tabset-1-2-tab" data-bs-toggle="tab" data-bs-target="#tabset-1-2" aria-controls="tabset-1-2" aria-selected="false" href="">Julia (DataFrames.jl)</a></li><li class="nav-item"><a class="nav-link" id="tabset-1-3-tab" data-bs-toggle="tab" data-bs-target="#tabset-1-3" aria-controls="tabset-1-3" aria-selected="false" href="">Python (pandas)</a></li><li class="nav-item"><a class="nav-link" id="tabset-1-4-tab" data-bs-toggle="tab" data-bs-target="#tabset-1-4" aria-controls="tabset-1-4" aria-selected="false" href="">Python (polars)</a></li><li class="nav-item"><a class="nav-link" id="tabset-1-5-tab" data-bs-toggle="tab" data-bs-target="#tabset-1-5" aria-controls="tabset-1-5" aria-selected="false" href="">Python (Naive way)</a></li></ul>
<div class="tab-content">
<div id="tabset-1-1" class="tab-pane active" aria-labelledby="tabset-1-1-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb1" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb1-1"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">library</span>(dplyr)</span>
<span id="cb1-2"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">library</span>(tidyr)</span>
<span id="cb1-3"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">library</span>(purrr)</span>
<span id="cb1-4"></span>
<span id="cb1-5">hugeQuantumSys <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">function</span>(x1, x2, x3) {</span>
<span id="cb1-6">    <span class="do" style="color: #5E5E5E;
background-color: null;
font-style: italic;">## Some huge calculations</span></span>
<span id="cb1-7">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">return</span>(<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">c</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">y1 =</span> x1 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> x2 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> x3, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">y2 =</span> x1 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> x2 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> x3, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">y3 =</span> x1 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span> x2 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span> x3))</span>
<span id="cb1-8">}</span>
<span id="cb1-9"></span>
<span id="cb1-10">df <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">expand_grid</span>(</span>
<span id="cb1-11">    <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">x1 =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">seq</span>(<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">length =</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">100</span>),</span>
<span id="cb1-12">    <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">x2 =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">seq</span>(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>, pi <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">/</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">length =</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">100</span>),</span>
<span id="cb1-13">    <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">x3 =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">seq</span>(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">length =</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">100</span>)</span>
<span id="cb1-14">) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|&gt;</span></span>
<span id="cb1-15">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">mutate</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">results =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">pmap</span>(<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">list</span>(x1, x2, x3), hugeQuantumSys)) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|&gt;</span></span>
<span id="cb1-16">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">unnest_wider</span>(results)</span>
<span id="cb1-17"></span>
<span id="cb1-18">arrow<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">::</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">write_parquet</span>(df, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"file.parquet"</span>)</span></code></pre></div></div>
</div>
<div id="tabset-1-2" class="tab-pane" aria-labelledby="tabset-1-2-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb2" style="background: #f1f3f5;"><pre class="sourceCode julia code-with-copy"><code class="sourceCode julia"><span id="cb2-1"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">using</span> <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">DataFrames</span></span>
<span id="cb2-2"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">using</span> <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">Arrow</span></span>
<span id="cb2-3"></span>
<span id="cb2-4"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">function</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">hugeQuantumSys</span>(x1, x2, x3)</span>
<span id="cb2-5">    <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Some huge calculations</span></span>
<span id="cb2-6">    <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">return</span> (y1 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> x1 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> x2 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> x3, y2 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> x1 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> x2 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> x3, y3 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> x1 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span> x2 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span> x3)</span>
<span id="cb2-7"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">end</span></span>
<span id="cb2-8"></span>
<span id="cb2-9">df <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">DataFrame</span>(</span>
<span id="cb2-10">    <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">Iterators</span>.<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">product</span>(</span>
<span id="cb2-11">        <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">range</span>(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>, <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>, length <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">100</span>),</span>
<span id="cb2-12">        <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">range</span>(<span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>, <span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">pi</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">/</span> <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>, length <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">100</span>),</span>
<span id="cb2-13">        <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">range</span>(<span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>, <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>, length <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">100</span>)</span>
<span id="cb2-14">    )</span>
<span id="cb2-15">)</span>
<span id="cb2-16"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">rename!</span>(df, [<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span>x1, <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span>x2, <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span>x3])</span>
<span id="cb2-17"></span>
<span id="cb2-18">y_vals <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">DataFrame</span>(<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">map</span>(hugeQuantumSys, df.x1, df.x2, df.x3))</span>
<span id="cb2-19"></span>
<span id="cb2-20">df <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">hcat</span>(df, y_vals)</span>
<span id="cb2-21"></span>
<span id="cb2-22">Arrow.<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">write</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"file.parquet"</span>, df)</span></code></pre></div></div>
</div>
<div id="tabset-1-3" class="tab-pane" aria-labelledby="tabset-1-3-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb3" style="background: #f1f3f5;"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb3-1"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">import</span> numpy <span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">as</span> np</span>
<span id="cb3-2"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">import</span> pandas <span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">as</span> pd</span>
<span id="cb3-3"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">import</span> itertools</span>
<span id="cb3-4"></span>
<span id="cb3-5"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">def</span> hugeQuantumSys(x1, x2, x3):</span>
<span id="cb3-6">    <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Some huge calculations</span></span>
<span id="cb3-7">    <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">return</span> {<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"y1"</span>: x1 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> x2 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> x3, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"y2"</span>: x1 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> x2 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> x3, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"y3"</span>: x1 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span> x2 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span> x3}</span>
<span id="cb3-8"></span>
<span id="cb3-9">df <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> pd.DataFrame(</span>
<span id="cb3-10">    itertools.product(</span>
<span id="cb3-11">        np.linspace(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">100</span>),</span>
<span id="cb3-12">        np.linspace(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>, np.pi <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">/</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">100</span>),</span>
<span id="cb3-13">        np.linspace(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">100</span>)</span>
<span id="cb3-14">    ),</span>
<span id="cb3-15">    columns<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span>[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"x1"</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"x2"</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"x3"</span>],</span>
<span id="cb3-16">)</span>
<span id="cb3-17"></span>
<span id="cb3-18">y_vals <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> df.<span class="bu" style="color: null;
background-color: null;
font-style: inherit;">apply</span>(</span>
<span id="cb3-19">    <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">lambda</span> row: hugeQuantumSys(row[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"x1"</span>], row[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"x2"</span>], row[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"x3"</span>]),</span>
<span id="cb3-20">    axis<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>,</span>
<span id="cb3-21">    result_type<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"expand"</span>,</span>
<span id="cb3-22">)</span>
<span id="cb3-23"></span>
<span id="cb3-24">df <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> pd.concat([df, y_vals], axis<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>)</span>
<span id="cb3-25"></span>
<span id="cb3-26">df.to_parquet(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"file.parquet"</span>)</span></code></pre></div></div>
</div>
<div id="tabset-1-4" class="tab-pane" aria-labelledby="tabset-1-4-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb4" style="background: #f1f3f5;"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb4-1"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">import</span> numpy <span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">as</span> np</span>
<span id="cb4-2"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">import</span> polars <span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">as</span> pl</span>
<span id="cb4-3"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">import</span> itertools</span>
<span id="cb4-4"></span>
<span id="cb4-5"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">def</span> hugeQuantumSys(x1, x2, x3):</span>
<span id="cb4-6">    <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Some huge calculations</span></span>
<span id="cb4-7">    <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">return</span> {<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"y1"</span>: x1 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> x2 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> x3, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"y2"</span>: x1 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> x2 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> x3, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"y3"</span>: x1 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span> x2 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span> x3}</span>
<span id="cb4-8"></span>
<span id="cb4-9">df <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> pl.DataFrame(</span>
<span id="cb4-10">    itertools.product(</span>
<span id="cb4-11">        np.linspace(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">100</span>),</span>
<span id="cb4-12">        np.linspace(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>, np.pi <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">/</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">100</span>),</span>
<span id="cb4-13">        np.linspace(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">100</span>)</span>
<span id="cb4-14">    ),</span>
<span id="cb4-15">    schema<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span>[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"x1"</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"x2"</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"x3"</span>],</span>
<span id="cb4-16">)</span>
<span id="cb4-17"></span>
<span id="cb4-18">y_vals <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> df.map_rows(<span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">lambda</span> row: hugeQuantumSys(row[<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>], row[<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>], row[<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>])).unnest()</span>
<span id="cb4-19"></span>
<span id="cb4-20">df <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> pl.concat([df, y_vals], how<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"horizontal"</span>)</span>
<span id="cb4-21"></span>
<span id="cb4-22">df.write_parquet(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"file.parquet"</span>)</span></code></pre></div></div>
</div>
<div id="tabset-1-5" class="tab-pane" aria-labelledby="tabset-1-5-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb5" style="background: #f1f3f5;"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb5-1"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">import</span> numpy <span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">as</span> np</span>
<span id="cb5-2"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">import</span> pandas <span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">as</span> pd</span>
<span id="cb5-3"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">import</span> itertools</span>
<span id="cb5-4"></span>
<span id="cb5-5"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">def</span> hugeQuantumSys(x1, x2, x3):</span>
<span id="cb5-6">    <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Some huge calculations</span></span>
<span id="cb5-7">    <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">return</span> x1 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> x2 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> x3, x1 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> x2 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> x3, x1 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span> x2 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span> x3</span>
<span id="cb5-8"></span>
<span id="cb5-9">x1 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> np.linspace(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">100</span>)</span>
<span id="cb5-10">x2 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> np.linspace(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>, np.pi <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">/</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">100</span>)</span>
<span id="cb5-11">x3 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> np.linspace(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">100</span>)</span>
<span id="cb5-12"></span>
<span id="cb5-13">N <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">len</span>(x1) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">len</span>(x2) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">len</span>(x3)</span>
<span id="cb5-14"></span>
<span id="cb5-15">x1_vals <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> np.zeros(N)</span>
<span id="cb5-16">x2_vals <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> np.zeros(N)</span>
<span id="cb5-17">x3_vals <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> np.zeros(N)</span>
<span id="cb5-18">y1_vals <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> np.zeros(N)</span>
<span id="cb5-19">y2_vals <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> np.zeros(N)</span>
<span id="cb5-20">y3_vals <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> np.zeros(N)</span>
<span id="cb5-21"></span>
<span id="cb5-22"><span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">for</span> i, (x1_val, x2_val, x3_val) <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">in</span> <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">enumerate</span>(itertools.product(x1, x2, x3)):</span>
<span id="cb5-23">    x1_vals[i] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> x1_val</span>
<span id="cb5-24">    x2_vals[i] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> x2_val</span>
<span id="cb5-25">    x3_vals[i] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> x3_val</span>
<span id="cb5-26">    y1_vals[i], y2_vals[i], y3_vals[i] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> hugeQuantumSys(x1_val, x2_val, x3_val)</span>
<span id="cb5-27"></span>
<span id="cb5-28">df <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> pd.DataFrame(</span>
<span id="cb5-29">    {</span>
<span id="cb5-30">        <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"x1"</span>: x1_vals,</span>
<span id="cb5-31">        <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"x2"</span>: x2_vals,</span>
<span id="cb5-32">        <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"x3"</span>: x3_vals,</span>
<span id="cb5-33">        <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"y1"</span>: y1_vals,</span>
<span id="cb5-34">        <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"y2"</span>: y2_vals,</span>
<span id="cb5-35">        <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"y3"</span>: y3_vals,</span>
<span id="cb5-36">    }</span>
<span id="cb5-37">)</span>
<span id="cb5-38"></span>
<span id="cb5-39">df.to_parquet(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"file.parquet"</span>)</span></code></pre></div></div>
</div>
</div>
</div>

<div class="no-row-height column-margin column-container"><div class="margin-aside">
<p>I’ve also included a “naive” way of doing this for Python, which should make it clear what we are trying to achieve.</p>
<p>Note that we have purposely let the function <code>hugeQuantumSys</code> return a named vector for tidyverse, named tuple for DataFrames.jl, and dictionaries for pandas and polars, so that they can be easily <em>unnested/expanded</em> with column names.</p>
<p>Also note that the return values for <code>y1</code>, <code>y2</code>, and <code>y3</code> are just examples.</p>
</div></div><p>Note that all the codes (except the naive approach) are essentially doing the same thing:</p>
<ol type="1">
<li>Create a dataframe with three columns with values that are Cartesian products<sup>2</sup> of <code>x1</code>, <code>x2</code>, and <code>x3</code>.</li>
<li>Create three new columns, <code>y1</code>, <code>y2</code>, and <code>y3</code>, which are the outputs of the function <code>hugeQuantumSys</code> (this can be done using higher-order functions such as <em>map</em> or <em>apply</em>, or list comprehension for Python and Julia).</li>
</ol>
<div class="no-row-height column-margin column-container"><div id="fn2"><p><sup>2</sup>&nbsp;E.g., if <code>x1 = [0, 1]</code> and <code>x2 = ["A", "B"]</code>, their Cartesian products are <code>(x1, x2) = [(0, "A"), (0, "B"), (1, "A"), (1, "B")]</code>. Basically all possible combinations.</p></div><div id="fn3"><p><sup>3</sup>&nbsp;This dataframe is not <a href="https://tidyr.tidyverse.org/articles/tidy-data.html"><em>“tidy”</em></a> per se <span class="citation" data-cites="WickhamTidy2014">(Wickham 2014)</span>, but it’s good enough for now.</p></div></div><p>Each row of the dataframe will thus be one <em>observation</em>, i.e., the <img src="https://latex.codecogs.com/png.latex?i">th row contains <img src="https://latex.codecogs.com/png.latex?x_%7B1,i%7D,%20x_%7B2,i%7D,%20x_%7B3,i%7D,%20y_%7B1,i%7D,%20y_%7B2,i%7D,%20y_%7B3,i%7D">, where <img src="https://latex.codecogs.com/png.latex?y_%7B1,i%7D,%20y_%7B2,i%7D,%20y_%7B3,i%7D%20=%20%5Cmathrm%7BhugeQuantumSys%7D(x_%7B1,i%7D,%20x_%7B2,i%7D,%20x_%7B3,i%7D)"><sup>3</sup>.</p>
<p>Now, I’m sure that most can agree that the syntax of tidyverse is much better than the others.</p>
<p>There’s a couple of reasons:</p>
<ol type="1">
<li><code>expand_grid</code> creates the dataframe <strong>and</strong> the Cartesian product in one go.</li>
<li>We can write <code>x1 = ...</code> directly to specify the column names.</li>
<li>We do not need to create a new dataframe <code>y_vals</code> and then concatenating it to the original dataframe <code>df</code>.</li>
<li>We can use the column names <code>x1</code>, <code>x2</code>, and <code>x3</code> directly without quotes, and without explicitly stating that we are referring to the rows.</li>
</ol>
<p>I think the magic happens at points 2, 3, 4, which are possible thanks to R’s <a href="http://adv-r.had.co.nz/Computing-on-the-language.html">non-standard evaluation</a>. Furthermore, pipes<sup>4</sup> makes the syntax cleaner where most workflows are just a long chaining of readable functions.</p>
<div class="no-row-height column-margin column-container"><div id="fn4"><p><sup>4</sup>&nbsp;The symbol <code>|&gt;</code> is the pipe operator, which can increase the readability of your code. The following are equivalent: <code>h(g(f(x, y), z))</code> and <code>x |&gt; f(y) |&gt; g(z) |&gt; h()</code>. Using pipes give us the clean syntax instead of writing <code>unnest_wider(mutate(expand_grid(...), ...), ...)</code></p></div></div></section>
<section id="clean-sql-like-operations" class="level2 page-columns page-full">
<h2 class="anchored" data-anchor-id="clean-sql-like-operations">Clean SQL-like operations</h2>
<p>Supposed we want to:</p>
<ol type="1">
<li>look only at the case of <code>y3 &lt; 0</code>,</li>
<li>group into cases where <code>y1 * y2 &gt;= 0</code> and <code>y1 * y2 &lt; 0</code>,</li>
<li>find the mean values of <code>y1</code>, <code>y2</code>, and <code>y3</code> in each of these groups.</li>
</ol>
<div class="tabset-margin-container"></div><div class="panel-tabset">
<ul class="nav nav-tabs"><li class="nav-item"><a class="nav-link active" id="tabset-2-1-tab" data-bs-toggle="tab" data-bs-target="#tabset-2-1" aria-controls="tabset-2-1" aria-selected="true" href="">R (tidyverse)</a></li><li class="nav-item"><a class="nav-link" id="tabset-2-2-tab" data-bs-toggle="tab" data-bs-target="#tabset-2-2" aria-controls="tabset-2-2" aria-selected="false" href="">SQL</a></li><li class="nav-item"><a class="nav-link" id="tabset-2-3-tab" data-bs-toggle="tab" data-bs-target="#tabset-2-3" aria-controls="tabset-2-3" aria-selected="false" href="">Julia (DataFrames.jl)</a></li><li class="nav-item"><a class="nav-link" id="tabset-2-4-tab" data-bs-toggle="tab" data-bs-target="#tabset-2-4" aria-controls="tabset-2-4" aria-selected="false" href="">Julia (DataFramesMeta.jl)</a></li><li class="nav-item"><a class="nav-link" id="tabset-2-5-tab" data-bs-toggle="tab" data-bs-target="#tabset-2-5" aria-controls="tabset-2-5" aria-selected="false" href="">Python (pandas)</a></li><li class="nav-item"><a class="nav-link" id="tabset-2-6-tab" data-bs-toggle="tab" data-bs-target="#tabset-2-6" aria-controls="tabset-2-6" aria-selected="false" href="">Python (polars)</a></li></ul>
<div class="tab-content">
<div id="tabset-2-1" class="tab-pane active" aria-labelledby="tabset-2-1-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb6" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb6-1">df <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|&gt;</span></span>
<span id="cb6-2">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">filter</span>(y3 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|&gt;</span></span>
<span id="cb6-3">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">mutate</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">is_positive =</span> (y1 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> y2 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;=</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>)) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|&gt;</span></span>
<span id="cb6-4">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">group_by</span>(is_positive) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|&gt;</span></span>
<span id="cb6-5">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">summarize</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">y1_mean =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">mean</span>(y1),</span>
<span id="cb6-6">            <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">y2_mean =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">mean</span>(y2),</span>
<span id="cb6-7">            <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">y3_mean =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">mean</span>(y3))</span></code></pre></div></div>
</div>
<div id="tabset-2-2" class="tab-pane" aria-labelledby="tabset-2-2-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb7" style="background: #f1f3f5;"><pre class="sourceCode sql code-with-copy"><code class="sourceCode sql"><span id="cb7-1"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">SELECT</span></span>
<span id="cb7-2">    (y1 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> y2) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;=</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">AS</span> is_positive,</span>
<span id="cb7-3">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">AVG</span>(y1) <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">AS</span> y1_mean,</span>
<span id="cb7-4">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">AVG</span>(y2) <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">AS</span> y2_mean,</span>
<span id="cb7-5">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">AVG</span>(y3) <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">AS</span> y3_mean</span>
<span id="cb7-6"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">FROM</span> df</span>
<span id="cb7-7"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">WHERE</span> y3 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span></span>
<span id="cb7-8"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">GROUP</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">BY</span> (y1 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> y2) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;=</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>;</span></code></pre></div></div>
</div>
<div id="tabset-2-3" class="tab-pane" aria-labelledby="tabset-2-3-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb8" style="background: #f1f3f5;"><pre class="sourceCode julia code-with-copy"><code class="sourceCode julia"><span id="cb8-1"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">using</span> <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">Statistics</span></span>
<span id="cb8-2"></span>
<span id="cb8-3">df <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|&gt;</span></span>
<span id="cb8-4">    x <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-&gt;</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">filter</span>(row <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-&gt;</span> row.y3 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span> <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>, x) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|&gt;</span></span>
<span id="cb8-5">    x <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-&gt;</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">transform</span>(x, [<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span>y1, <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span>y2] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=&gt;</span> ((y1, y2) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-&gt;</span> y1 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.*</span> y2 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.&gt;=</span> <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=&gt;</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span>is_positive) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|&gt;</span></span>
<span id="cb8-6">    x <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-&gt;</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">combine</span>(</span>
<span id="cb8-7">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">groupby</span>(x, <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span>is_positive),</span>
<span id="cb8-8">    <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span>y1 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=&gt;</span> mean <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=&gt;</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span>y1_mean,</span>
<span id="cb8-9">    <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span>y2 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=&gt;</span> mean <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=&gt;</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span>y2_mean,</span>
<span id="cb8-10">    <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span>y3 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=&gt;</span> mean <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=&gt;</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span>y3_mean</span>
<span id="cb8-11">)</span>
<span id="cb8-12"></span>
<span id="cb8-13"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># alternatively, without using pipes</span></span>
<span id="cb8-14"></span>
<span id="cb8-15">df_filtered <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">filter</span>(row <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-&gt;</span> row.y3 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span> <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>, df)</span>
<span id="cb8-16">df_mutated <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">transform</span>(df_filtered, [<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span>y1, <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span>y2] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=&gt;</span> ((y1, y2) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-&gt;</span> y1 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.*</span> y2 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.&gt;=</span> <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=&gt;</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span>is_positive)</span>
<span id="cb8-17"></span>
<span id="cb8-18"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">combine</span>(</span>
<span id="cb8-19">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">groupby</span>(df_mutated, <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span>is_positive),</span>
<span id="cb8-20">    <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span>y1 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=&gt;</span> mean <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=&gt;</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span>y1_mean,</span>
<span id="cb8-21">    <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span>y2 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=&gt;</span> mean <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=&gt;</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span>y2_mean,</span>
<span id="cb8-22">    <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span>y3 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=&gt;</span> mean <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=&gt;</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span>y3_mean</span>
<span id="cb8-23">)</span></code></pre></div></div>
</div>
<div id="tabset-2-4" class="tab-pane" aria-labelledby="tabset-2-4-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb9" style="background: #f1f3f5;"><pre class="sourceCode julia code-with-copy"><code class="sourceCode julia"><span id="cb9-1"><span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">@chain</span> df <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">begin</span></span>
<span id="cb9-2">    <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">@subset</span>(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span>y3 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.&lt;</span> <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>)</span>
<span id="cb9-3">    <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">@transform</span>(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span>is_positive <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> (<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span>y1 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.*</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span>y2 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.&gt;=</span> <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>))</span>
<span id="cb9-4">    <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">@groupby</span>(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span>is_positive)</span>
<span id="cb9-5">    <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">@combine</span>(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span>y1_mean <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">mean</span>(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span>y1),</span>
<span id="cb9-6">             <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span>y2_mean <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">mean</span>(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span>y2),</span>
<span id="cb9-7">             <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span>y3_mean <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">mean</span>(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span>y3))</span>
<span id="cb9-8"><span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">end</span></span></code></pre></div></div>
</div>
<div id="tabset-2-5" class="tab-pane" aria-labelledby="tabset-2-5-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb10" style="background: #f1f3f5;"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb10-1">(</span>
<span id="cb10-2">    df.query(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"y3 &lt; 0"</span>)</span>
<span id="cb10-3">    .assign(is_positive<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">lambda</span> x: (x[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"y1"</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> x[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"y2"</span>]) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;=</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>)</span>
<span id="cb10-4">    .groupby(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"is_positive"</span>)</span>
<span id="cb10-5">    .agg(</span>
<span id="cb10-6">        y1_mean<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"y1"</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"mean"</span>),</span>
<span id="cb10-7">        y2_mean<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"y2"</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"mean"</span>),</span>
<span id="cb10-8">        y3_mean<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"y3"</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"mean"</span>)</span>
<span id="cb10-9">    )</span>
<span id="cb10-10">    .reset_index()</span>
<span id="cb10-11">)</span></code></pre></div></div>
</div>
<div id="tabset-2-6" class="tab-pane" aria-labelledby="tabset-2-6-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb11" style="background: #f1f3f5;"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb11-1">(</span>
<span id="cb11-2">    df.<span class="bu" style="color: null;
background-color: null;
font-style: inherit;">filter</span>(pl.col(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"y3"</span>) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>)</span>
<span id="cb11-3">    .with_columns(is_positive<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span>(pl.col(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"y1"</span>) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> pl.col(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"y2"</span>)) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;=</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>)</span>
<span id="cb11-4">    .group_by(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"is_positive"</span>)</span>
<span id="cb11-5">    .agg(</span>
<span id="cb11-6">        y1_mean<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span>pl.col(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"y1"</span>).mean(),</span>
<span id="cb11-7">        y2_mean<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span>pl.col(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"y2"</span>).mean(),</span>
<span id="cb11-8">        y3_mean<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span>pl.col(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"y3"</span>).mean(),</span>
<span id="cb11-9">    )</span>
<span id="cb11-10">)</span></code></pre></div></div>
</div>
</div>
</div>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb12" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb12-1"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># A tibble: 2 × 4</span></span>
<span id="cb12-2">  is_positive y1_mean y2_mean y3_mean</span>
<span id="cb12-3">  <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span>lgl<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;</span>         <span class="er" style="color: #AD0000;
background-color: null;
font-style: inherit;">&lt;</span>dbl<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;</span>   <span class="er" style="color: #AD0000;
background-color: null;
font-style: inherit;">&lt;</span>dbl<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;</span>   <span class="er" style="color: #AD0000;
background-color: null;
font-style: inherit;">&lt;</span>dbl<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;</span></span>
<span id="cb12-4"><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span> <span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">FALSE</span>         <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0.915</span>  <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0.221</span>  <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">1.87</span></span>
<span id="cb12-5"><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span> <span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">TRUE</span>          <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">1.61</span>    <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0.187</span>  <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0.950</span></span></code></pre></div></div>
<p>I think R’s non-standard evaluation and pipes really shines here. First, Python’s lack of non-standard evaluation means that we have to specify anonymous functions, <code>pl.col</code>, or quotes when accessing the rows and columns<sup>5</sup>. Second, Julia’s not so well-designed pipes means that we have to write anonymous functions for each pipe (i.e., the annoying <code>x -&gt; ...</code> notation). Though thankfully, <a href="https://juliadata.org/DataFramesMeta.jl/stable/">DataFramesMeta.jl</a> uses <a href="https://docs.julialang.org/en/v1/manual/metaprogramming/">macros</a> to achieve a syntax closer to tidyverse<sup>6</sup>.</p>
<div class="no-row-height column-margin column-container"><div id="fn5"><p><sup>5</sup>&nbsp;Still, Python’s dot notation really saves the day here to make it readable, though we have to surround them with parenthesis.</p></div><div id="fn6"><p><sup>6</sup>&nbsp;Additionally, the existence of <a href="https://docs.julialang.org/en/v1/manual/metaprogramming/#Symbols"><em>symbols</em></a> in Julia, due to partly being inspired by Lisp, allows us to write <code>:</code> instead of quotes to refer to columns, which is a little better I guess?</p></div></div><p>The DataFrames.jl docs has a good comparison <a href="https://dataframes.juliadata.org/stable/man/comparisons/">here</a>.</p>
</section>
<section id="grammar-of-graphics" class="level2 page-columns page-full">
<h2 class="anchored" data-anchor-id="grammar-of-graphics">Grammar of graphics</h2>
<p>Next, supposed we want to plot a heat map <code>y1</code> against <code>x1</code> and <code>x3</code> for each <code>x2 = 0</code>, <code>x2 = pi/6</code>, <code>x2 = pi/3</code>, and <code>x2 = pi/2</code>, i.e., we have four subplots for the four different values of <code>x2</code>.</p>
<div class="tabset-margin-container"></div><div class="panel-tabset">
<ul class="nav nav-tabs"><li class="nav-item"><a class="nav-link active" id="tabset-3-1-tab" data-bs-toggle="tab" data-bs-target="#tabset-3-1" aria-controls="tabset-3-1" aria-selected="true" href="">R (ggplot2)</a></li><li class="nav-item"><a class="nav-link" id="tabset-3-2-tab" data-bs-toggle="tab" data-bs-target="#tabset-3-2" aria-controls="tabset-3-2" aria-selected="false" href="">Python (Matplotlib)</a></li><li class="nav-item"><a class="nav-link" id="tabset-3-3-tab" data-bs-toggle="tab" data-bs-target="#tabset-3-3" aria-controls="tabset-3-3" aria-selected="false" href="">Julia (Makie.jl)</a></li></ul>
<div class="tab-content">
<div id="tabset-3-1" class="tab-pane active" aria-labelledby="tabset-3-1-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb13" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb13-1"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">library</span>(ggplot2)</span>
<span id="cb13-2"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">library</span>(ggtypst)</span>
<span id="cb13-3"></span>
<span id="cb13-4">df <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|&gt;</span></span>
<span id="cb13-5">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">filter</span>(x2 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">==</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span> <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">near</span>(x2, pi <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">/</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">6</span>) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">near</span>(x2, pi <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">/</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">3</span>) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">near</span>(x2, pi <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">/</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>)) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|&gt;</span></span>
<span id="cb13-6">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">mutate</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">x2 =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">paste</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"x_2 ="</span>, <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">signif</span>(x2 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">/</span> pi, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">3</span>), <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"pi"</span>)) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|&gt;</span></span>
<span id="cb13-7">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">ggplot</span>(<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">aes</span>(x1, x3, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">fill =</span> y1)) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb13-8">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">geom_raster</span>() <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb13-9">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">scale_fill_steps</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">n.breaks =</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">10</span>) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb13-10">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">facet_wrap</span>(<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">~</span>x2) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb13-11">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">theme</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">strip.text =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">element_math_typst</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">size =</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">12</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">face =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"bold"</span>))</span></code></pre></div></div>
</div>
<div id="tabset-3-2" class="tab-pane" aria-labelledby="tabset-3-2-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb14" style="background: #f1f3f5;"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb14-1"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">import</span> matplotlib.pyplot <span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">as</span> plt</span>
<span id="cb14-2"></span>
<span id="cb14-3">filtered_df <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> df[</span>
<span id="cb14-4">    (df[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"x2"</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">==</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>)</span>
<span id="cb14-5">    <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|</span> (np.isclose(df[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"x2"</span>], np.pi <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">/</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">6</span>))</span>
<span id="cb14-6">    <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|</span> (np.isclose(df[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"x2"</span>], np.pi <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">/</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">3</span>))</span>
<span id="cb14-7">    <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|</span> (np.isclose(df[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"x2"</span>], np.pi <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">/</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>))</span>
<span id="cb14-8">].copy()</span>
<span id="cb14-9"></span>
<span id="cb14-10">filtered_df[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"x2"</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> filtered_df[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"x2"</span>].<span class="bu" style="color: null;
background-color: null;
font-style: inherit;">apply</span>(<span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">lambda</span> x: <span class="ss" style="color: #20794D;
background-color: null;
font-style: inherit;">f"x2 = </span><span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span>x <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">/</span> np<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>pi<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:.3f}</span><span class="ss" style="color: #20794D;
background-color: null;
font-style: inherit;"> pi"</span>)</span>
<span id="cb14-11">x2_groups <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> filtered_df.groupby(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"x2"</span>, sort<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">False</span>)</span>
<span id="cb14-12"></span>
<span id="cb14-13">fig, axes <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> plt.subplots(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>)</span>
<span id="cb14-14"></span>
<span id="cb14-15"><span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">for</span> ax, (x2_label, group) <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">in</span> <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">zip</span>(axes.flatten(), x2_groups):</span>
<span id="cb14-16">    x1_unique <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> np.sort(group[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"x1"</span>].unique())</span>
<span id="cb14-17">    x3_unique <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> np.sort(group[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"x3"</span>].unique())</span>
<span id="cb14-18"></span>
<span id="cb14-19">    X1, X3 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> np.meshgrid(x1_unique, x3_unique)</span>
<span id="cb14-20"></span>
<span id="cb14-21">    Z <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> group.pivot_table(index<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"x3"</span>, columns<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"x1"</span>, values<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"y1"</span>).values</span>
<span id="cb14-22"></span>
<span id="cb14-23">    contourf <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> ax.contourf(X1, X3, Z)</span>
<span id="cb14-24">    plt.colorbar(contourf, ax<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span>ax)</span>
<span id="cb14-25"></span>
<span id="cb14-26">    ax.set_xlabel(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"x1"</span>)</span>
<span id="cb14-27">    ax.set_ylabel(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"x3"</span>)</span>
<span id="cb14-28">    ax.set_title(x2_label)</span>
<span id="cb14-29"></span>
<span id="cb14-30">plt.tight_layout()</span>
<span id="cb14-31">plt.show()</span></code></pre></div></div>
</div>
<div id="tabset-3-3" class="tab-pane" aria-labelledby="tabset-3-3-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb15" style="background: #f1f3f5;"><pre class="sourceCode julia code-with-copy"><code class="sourceCode julia"><span id="cb15-1"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">using</span> <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">CairoMakie</span></span>
<span id="cb15-2"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">using</span> <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">Printf</span></span>
<span id="cb15-3"></span>
<span id="cb15-4">filtered_df <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> df[</span>
<span id="cb15-5">    (df.x2 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.==</span> <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.|</span></span>
<span id="cb15-6">        (<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">isapprox</span>.(df.x2, <span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">π</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">/</span> <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">6</span>)) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.|</span></span>
<span id="cb15-7">        (<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">isapprox</span>.(df.x2, <span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">π</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">/</span> <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">3</span>)) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.|</span></span>
<span id="cb15-8">        (<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">isapprox</span>.(df.x2, <span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">π</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">/</span> <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>)),</span>
<span id="cb15-9">    <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span>,</span>
<span id="cb15-10">]</span>
<span id="cb15-11"></span>
<span id="cb15-12">filtered_df.x2_label <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> [<span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">@sprintf</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"x2 = %.3f pi"</span>, x <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">/</span> <span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">π</span>) for x <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">in</span> filtered_df.x2]</span>
<span id="cb15-13">x2_groups <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">groupby</span>(filtered_df, <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span>x2_label)</span>
<span id="cb15-14"></span>
<span id="cb15-15">fig <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">Figure</span>()</span>
<span id="cb15-16"></span>
<span id="cb15-17"><span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">for</span> (idx, group) <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">in</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">enumerate</span>(x2_groups)</span>
<span id="cb15-18">    row <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">div</span>(idx <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span> <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>, <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span></span>
<span id="cb15-19">    col <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">mod</span>(idx <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span> <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>, <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span></span>
<span id="cb15-20"></span>
<span id="cb15-21">    ax <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">Axis</span>(fig[row, col], xlabel <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"x1"</span>, ylabel <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"x3"</span>, title <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> group.x2_label[<span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>])</span>
<span id="cb15-22"></span>
<span id="cb15-23">    x1_unique <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">sort</span>(<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">unique</span>(group.x1))</span>
<span id="cb15-24">    x3_unique <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">sort</span>(<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">unique</span>(group.x3))</span>
<span id="cb15-25"></span>
<span id="cb15-26">    X1 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">repeat</span>(x1_unique<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">'</span>, <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">length</span>(x3_unique), <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>)</span>
<span id="cb15-27">    X3 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">repeat</span>(x3_unique, <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>, <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">length</span>(x1_unique))</span>
<span id="cb15-28"></span>
<span id="cb15-29">    Z <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">fill</span>(<span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">NaN</span>, <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">length</span>(x3_unique), <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">length</span>(x1_unique))</span>
<span id="cb15-30">    <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">for</span> (i, x3_val) <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">in</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">enumerate</span>(x3_unique), (j, x1_val) <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">in</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">enumerate</span>(x1_unique)</span>
<span id="cb15-31">        row_match <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">findfirst</span>((group.x1 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.≈</span> x1_val) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.&amp;</span> (group.x3 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.≈</span> x3_val))</span>
<span id="cb15-32">        <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">if</span> !<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">isnothing</span>(row_match)</span>
<span id="cb15-33">            Z[i, j] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> group.y1[row_match]</span>
<span id="cb15-34">        <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">end</span></span>
<span id="cb15-35">    <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">end</span></span>
<span id="cb15-36"></span>
<span id="cb15-37">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">contourf!</span>(ax, X1, X3, Z, levels <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">15</span>)</span>
<span id="cb15-38"><span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">end</span></span>
<span id="cb15-39"></span>
<span id="cb15-40">fig</span></code></pre></div></div>
</div>
</div>
</div>
<div class="quarto-figure quarto-figure-center">
<figure class="figure">
<p><img src="https://jianweicheong.codeberg.page/media/posts/r-heart/ggplot.png" class="img-fluid figure-img"></p>
<figcaption>Plot by <a href="https://ggplot2.tidyverse.org/">ggplot2</a>.</figcaption>
</figure>
</div>
<p>I don’t think anything else can beat this<sup>7</sup>. The <a href="https://en.wikipedia.org/wiki/Wilkinson%27s_Grammar_of_Graphics">grammar of graphics</a> style of plotting works nicely on tidy data. We just have to specify the <em>aesthetics</em> for what to plot, <em>geoms</em> for how to plot, and <em>facets</em> for the what to subplot.</p>
<div class="no-row-height column-margin column-container"><div id="fn7"><p><sup>7</sup>&nbsp;Not counting plotting libraries that aims to replicate ggplot2 of course. Though, even then it is difficult to come close.</p></div></div><p>Also, non-standard evaluation shines again as we see that it knows to label the x and y labels as <code>x1</code> and <code>x3</code> based on the column names. We also easily <em>mutated</em> <code>x2</code> to include <code>x_2 =</code> and <code>pi</code> without anonymous functions, and showcased <a href="https://yousa-mirage.github.io/ggtypst/">ggtypst</a> which converted them to math using <a href="../posts/tools-academic.html#typst">Typst</a>.</p>
<p>I also have to confess that the Matplotlib and Makie.jl codes are generated by Claude Haiku 4.5. I’ve <a href="../posts/typst-lilaq.html">mentioned</a> how much of a chore plotting is. Ain’t nobody has time for the Python and Julia codes above.</p>
</section>
<section id="tidyverse-with-different-backends" class="level2 page-columns page-full">
<h2 class="anchored" data-anchor-id="tidyverse-with-different-backends">Tidyverse with different backends</h2>
<p>Unfortunately, the tidyverse (using <a href="https://dplyr.tidyverse.org/">dplyr</a>/<a href="https://tidyr.tidyverse.org/">tidyr</a>) and pandas codes are actually quite slow, particularly, we are acting on <img src="https://latex.codecogs.com/png.latex?100%20%5Ctimes%20100%20%5Ctimes%20100%20=%201000000"> rows, i.e., the number of Cartesian products for <code>x1</code>, <code>x2</code>, and <code>x3</code>.</p>
<p>This is one of the reasons why polars and <a href="https://en.wikipedia.org/wiki/DuckDB">DuckDB</a> are now getting more popular over pandas for data science in Python due to their speed.</p>
<p>But it’s quite a chore to convert your pandas code to polars for example: you’ll have to change <code>columns</code> to <code>schema</code>, <code>apply</code> to <code>map_rows</code>, <code>result_type='expand'</code> to <code>.unnest()</code>, <code>axis=1</code> to <code>how='horizontal'</code>, <code>query</code> to <code>filter</code>, <code>assign</code> to <code>with_columns</code> etc.</p>
<p>What about R? There’s a <a href="https://pola-rs.github.io/r-polars/">polars package for R</a>, how hard is it to switch?</p>
<p>Because the tidyverse workflow and syntax is such a joy to use, how about we keep using the tidyverse code, but just let it run polars in the background? That’s exactly what <a href="https://www.tidypolars.etiennebacher.com/">tidypolars</a> does!</p>
<p>And it’s not the only one. There’s <a href="https://duckplyr.tidyverse.org/">duckplyr</a> for <a href="https://en.wikipedia.org/wiki/DuckDB">DuckDB</a>, <a href="https://dbplyr.tidyverse.org/">dbplyr</a> for <a href="https://en.wikipedia.org/wiki/SQL">SQL</a>, <a href="https://arrow.apache.org/docs/r/index.html">arrow</a> for <a href="https://en.wikipedia.org/wiki/Apache_Arrow">Apache Arrow</a>, <a href="https://spark.posit.co/">sparklyr</a> for <a href="https://en.wikipedia.org/wiki/Apache_Spark">Apache Spark</a>, and <a href="https://dtplyr.tidyverse.org/">dtplyr</a> or <a href="https://markfairbanks.github.io/tidytable/">tidytable</a> for <a href="https://rdatatable.gitlab.io/data.table/">data.table</a>.</p>
<p>Let’s give tidytable a try.</p>
<div class="tabset-margin-container"></div><div class="panel-tabset">
<ul class="nav nav-tabs"><li class="nav-item"><a class="nav-link active" id="tabset-4-1-tab" data-bs-toggle="tab" data-bs-target="#tabset-4-1" aria-controls="tabset-4-1" aria-selected="true" href="">R (dplyr/tidyr)</a></li><li class="nav-item"><a class="nav-link" id="tabset-4-2-tab" data-bs-toggle="tab" data-bs-target="#tabset-4-2" aria-controls="tabset-4-2" aria-selected="false" href="">R (tidytable)</a></li></ul>
<div class="tab-content">
<div id="tabset-4-1" class="tab-pane active" aria-labelledby="tabset-4-1-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb16" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb16-1"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">library</span>(dplyr)</span>
<span id="cb16-2"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">library</span>(tidyr)</span>
<span id="cb16-3"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">library</span>(purrr)</span>
<span id="cb16-4"></span>
<span id="cb16-5">hugeQuantumSys <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">function</span>(x1, x2, x3) {</span>
<span id="cb16-6">    <span class="do" style="color: #5E5E5E;
background-color: null;
font-style: italic;">## Some huge calculations</span></span>
<span id="cb16-7">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">return</span>(<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">c</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">y1 =</span> x1 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> x2 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> x3, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">y2 =</span> x1 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> x2 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> x3, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">y3 =</span> x1 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span> x2 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span> x3))</span>
<span id="cb16-8">}</span>
<span id="cb16-9"></span>
<span id="cb16-10">df <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">expand_grid</span>(</span>
<span id="cb16-11">    <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">x1 =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">seq</span>(<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">length =</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">100</span>),</span>
<span id="cb16-12">    <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">x2 =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">seq</span>(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>, pi <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">/</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">length =</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">100</span>),</span>
<span id="cb16-13">    <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">x3 =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">seq</span>(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">length =</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">100</span>)</span>
<span id="cb16-14">) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|&gt;</span></span>
<span id="cb16-15">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">mutate</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">results =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">pmap</span>(<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">list</span>(x1, x2, x3), hugeQuantumSys)) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|&gt;</span></span>
<span id="cb16-16">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">unnest_wider</span>(results)</span></code></pre></div></div>
</div>
<div id="tabset-4-2" class="tab-pane" aria-labelledby="tabset-4-2-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb17" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb17-1"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">library</span>(tidytable)</span>
<span id="cb17-2"></span>
<span id="cb17-3">hugeQuantumSys <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">function</span>(x1, x2, x3) {</span>
<span id="cb17-4">    <span class="do" style="color: #5E5E5E;
background-color: null;
font-style: italic;">## Some huge calculations</span></span>
<span id="cb17-5">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">return</span>(<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">c</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">y1 =</span> x1 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> x2 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> x3, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">y2 =</span> x1 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> x2 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> x3, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">y3 =</span> x1 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span> x2 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span> x3))</span>
<span id="cb17-6">}</span>
<span id="cb17-7"></span>
<span id="cb17-8">df <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">expand_grid</span>(</span>
<span id="cb17-9">    <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">x1 =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">seq</span>(<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">length =</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">100</span>),</span>
<span id="cb17-10">    <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">x2 =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">seq</span>(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>, pi <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">/</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">length =</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">100</span>),</span>
<span id="cb17-11">    <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">x3 =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">seq</span>(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">length =</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">100</span>)</span>
<span id="cb17-12">) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|&gt;</span></span>
<span id="cb17-13">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">mutate</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">results =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">pmap</span>(<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">list</span>(x1, x2, x3), hugeQuantumSys)) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|&gt;</span></span>
<span id="cb17-14">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">unnest_wider</span>(results)</span></code></pre></div></div>
</div>
</div>
</div>
<p>See any differences between the two codes? There isn’t any as tidytable is a drop-in replacement for tidyverse’s <a href="https://dplyr.tidyverse.org/">dplyr</a>, <a href="https://tidyr.tidyverse.org/">tidyr</a>, and <a href="https://purrr.tidyverse.org/">purrr</a>, but now uses data.table as the backend (instead of tidyverse’s <a href="https://tibble.tidyverse.org/">tibble</a> or R’s default data.frame.</p>
<p>But what if <code>hugeQuantumSys</code> is the one taking a long time to run? How do I parallelize its run on the 1000000 rows? Simple, use <a href="https://furrr.futureverse.org/">furrr</a> instead of purrr, and replace <code>pmap</code> with <code>future_pmap</code>.</p>
<p>You see, the tidyverse has become such a staple in R that a lot of packages are designed around it, which makes integration with it so simple<sup>8</sup>.</p>
<div class="no-row-height column-margin column-container"><div id="fn8"><p><sup>8</sup>&nbsp;This reminds me of Julia’s <a href="https://sciml.ai/">SciML</a>, where different packages all works together thanks to Julia’s <a href="https://en.wikipedia.org/wiki/Multiple_dispatch">multiple dispatch</a>. Since we are talking about parallelization, another shout out to Julia is the ability to allow one to multithread their for loops just by adding the <code>Threads.@threads</code> macros in front of it, e.g., <code>Threads.@threads for x in 0:1000 ...</code>.</p></div></div><p>Here’s a little benchmark for the above codes. For more accurate benchmarks, refer to <a href="https://duckdblabs.github.io/db-benchmark/">this</a>.</p>
<div class="cell">
<div class="cell-output-display">
<style>
th, td{ padding: 5px; }
.lt-table {
  --bg: 4px solid transparent;

  border-collapse: separate;
  caption-side: top;
  margin: 1em auto;
  border-spacing: 0;
  border-top: 1.5px solid #888;
  border-bottom: 1.5px solid #888;

  caption {
    padding: .5em;
    text-align: center;
    .lt-title    { font-size: 1.2em; }
    .lt-subtitle { font-size: 0.9em; }
  }

  th { font-weight: normal; }
  tbody {
    :not(:last-child) :is(td, th) { border-bottom: 1px   solid #ddd; }
    .lt-row-group :is(td, th)     { border-bottom: 1.5px solid #ddd; }
    .lt-row-group { text-align: left; }
    .lt-row-group { vertical-align: top; }
  }
  thead th {
    border-bottom: 1px solid #888;
    &:not(:first-child) { border-left: var(--bg); }
    &:not(:last-child) { border-right: var(--bg); }
  }
  .lt-indent { padding-left: 1em; }
  .lt-spanner-row {
    .lt-spanner       { border-bottom: 1px solid #bbb; text-align: center; }
    .lt-spanner-empty { border-bottom: none; }
  }

  .lt-footer {
    font-size: 0.85em;
    .lt-footnote:first-child td { border-top: 1px solid #aaa; }
    .lt-source-note:first-child td { border-top: 1px dashed #aaa; }
    .lt-source-note td { font-style: italic; }
  }

  .al-r { text-align: right; white-space: nowrap; }
  .al-c { text-align: center; }

  .lt-fnref {
    color: #06c;
    &::before { content: " ["; }
    &::after  { content: "]"; }
  }
}
</style>
<script>((window.LT=window.LT||{}).q=window.LT.q||[]).push({s:document.currentScript,d:
{
  "data": {
    "Language": ["Julia", "Julia", "Python", "Python", "R", "R", "Python", "R"],
    "Package": ["DataFrames.jl (subsequent runs)", "DataFrames.jl (1st run)", "Naive way", "polars", "data.table", "tidytable", "pandas", "dplyr & tidyr"],
    "Median (seconds)": [0.0576, 0.295, 0.8925, 1.2, 2.81, 11.855, 13.19, 24.08],
    "Standard deviation": [0.0148, 0.00756, 0.0165, 0.0457, 0.0536, 0.0534, 0.246, 0.316]
  },
  "footnotes": [
    {
      "text": "Sample size = 10 each.",
      "location": {
        "type": "column_labels",
        "columns": ["Median (seconds)"]
      }
    },
    {
      "text": "Julia precompiles functions, making them faster for subsequent runs. \"(1st run)\" refers to runs without precompilation.",
      "location": {
        "type": "body",
        "columns": ["Package"],
        "rows": [1, 2]
      }
    },
    {
      "text": "Note that there are polars packages for R, Julia, and Rust, but Python was used for this benchmark.",
      "location": {
        "type": "body",
        "columns": ["Package"],
        "rows": [4]
      }
    }
  ]
}
})</script>
<script>
/* lt.js — build a semantic <table> from a JSON spec.
 * Call LT.build(spec) from an inline <script> to render a table in place.
 * One runtime per page renders any number of tables.
 */
(root => {
  "use strict";
  if (root.LT?.buildHtml) return;  // duplicate inclusion is a no-op

  // `[<]` (not `<`) avoids `</…` so this file is safe to inline in <script>.
  const esc = s => String(s)
    .replace(/&/g, "&amp;").replace(/[<]/g, "&lt;")
    .replace(/>/g, "&gt;").replace(/"/g, "&quot;");
  const sup = i => `<sup class="lt-fnref">${i}</sup>`;
  // Stringify, mapping null/undefined to "" (0 and false stringify normally).
  const str = v => String(v ?? "");

  // --- Number formatting ---
  const isNum = v => typeof v === "number" && !isNaN(v);
  // A column is "numeric" if its first non-null value is a number.
  const numCol = col => col?.length && typeof col.find(v => v != null) === "number";

  function fmtNumber(v, decimals, bigMark) {
    if (!isNum(v)) return null;
    let s = decimals != null ? v.toFixed(decimals) : String(v);
    if (/^-0(\.0+)?$/.test(s)) s = s.slice(1);
    if (bigMark) {
      const parts = s.split(".");
      parts[0] = parts[0].replace(/\B(?=(\d{3})+(?!\d))/g, bigMark);
      s = parts.join(".");
    }
    if (s[0] === "-") s = "−" + s.slice(1);
    return decimals != null || bigMark ? s : null;
  }

  // --- Pattern merge ---
  function mergeVals(vals, pattern) {
    if (!pattern) return vals.filter(v => v !== "").join(" ");
    let remaining = pattern, result = "";
    while (remaining) {
      const m = remaining.match(/<<(.*?)>>/);
      if (!m) { result += subRefs(remaining, vals); break; }
      if (m.index > 0) result += subRefs(remaining.slice(0, m.index), vals);
      const block = m[1],
            refs = [...block.matchAll(/\{(\d+)\}/g)].map(x => +x[1]);
      if (!refs.length || refs.every(i => vals[i - 1] !== ""))
        result += subRefs(block, vals);
      remaining = remaining.slice(m.index + m[0].length);
    }
    return result;
  }
  function subRefs(s, vals) {
    for (let i = 0; i < vals.length; i++)
      s = s.split("{" + (i + 1) + "}").join(vals[i]);
    return s;
  }

  // --- Auto-format numeric columns ---
  function autoFmt(spec, display, nRow) {
    if (spec.auto_format === false) return;
    const data = spec.data || {}, ops = spec.ops || [],
          colNames = Object.keys(data);

    const fmtCols = new Set();
    for (const op of ops) {
      if (op.type === "fmt_number") for (const c of (op.columns || colNames)) fmtCols.add(c);
    }

    const getLabel = c => {
      for (const op of ops) {
        if (op.type === "label" && op.labels?.[c] != null) return op.labels[c];
      }
      return c;
    };

    for (const c of colNames) {
      if (fmtCols.has(c)) continue;
      const col = data[c];
      if (!numCol(col)) continue;

      const lbl = getLabel(c),
            pct = /%|[ _](pct|percent)$/i.test(lbl);

      if (/year/i.test(lbl) && col.every(v => v == null || /^\d{4}$/.test(String(v)))) continue;

      // Determine decimal places: find max significant decimals across column,
      // then cap dynamically based on the largest integer-part width (targeting
      // ~4 total significant digits): e.g., values <1 get up to 4 decimals,
      // 10-99 get 2, >=1000 get 0.
      let maxInt = 0, n = 0;
      for (const v of col) {
        if (!isNum(v)) continue;
        // Number of digits before the decimal point
        const a = Math.abs(pct ? v * 100 : v),
              intW = a < 1 ? 0 : Math.floor(Math.log10(a)) + 1;
        if (intW > maxInt) maxInt = intW;
        // Significant decimals (strip trailing zeros; adjust for pct *100)
        const m = String(v).match(/\.(\d+)/);
        if (!m) continue;
        const d = m[1].replace(/0+$/, "").length - (pct ? 2 : 0);
        if (d > n) n = d;
      }
      // Cap: 4 decimals max, minus integer width (so large numbers get fewer).
      // n is already >= 0, so only the upper cap is needed.
      n = Math.min(n, Math.max(4 - maxInt, 0));

      for (let i = 0; i < nRow; i++) {
        const v = col[i];
        if (!isNum(v)) continue;
        const s = fmtNumber(pct ? v * 100 : v, n, " ");
        if (s != null) display[c][i] = s + (pct ? "%" : "");
      }
    }
  }

  // --- Apply ops to data columns, return display strings per column ---
  function applyOps(spec) {
    const data = spec.data || {},
          ops = spec.ops || [],
          colNames = Object.keys(data),
          nRow = colNames.length ? data[colNames[0]].length : 0;

    // Working copy: display[col][row] = string
    const display = {};
    for (const c of colNames) {
      display[c] = data[c].map(str);
    }

    autoFmt(spec, display, nRow);

    // Run fn(col, row, rawValue) over each existing cell of the given columns.
    const eachCell = (cols, fn) => {
      for (const c of cols) {
        if (!data[c]) continue;
        for (let i = 0; i < nRow; i++) fn(c, i, data[c][i]);
      }
    };

    for (const op of ops) {
      const cols = op.columns || colNames;
      switch (op.type) {
        case "fmt_number":
          eachCell(cols, (c, i, raw) => {
            if (!isNum(raw)) return;
            const v = op.percent === true ? raw * 100 : raw,
                  psfx = op.percent ? "%" : "",
                  f = fmtNumber(v, op.decimals, op.big_mark ?? ""),
                  pfx = op.prefix || "", sfx = (op.suffix || "") + psfx;
            if (f != null) display[c][i] = pfx + f + sfx;
            else if (pfx || sfx) display[c][i] = pfx + String(v) + sfx;
          });
          break;
        case "fmt_date":
          eachCell(cols, (c, i, raw) => {
            if (raw == null) return;
            const d = raw instanceof Date ? raw : new Date(raw);
            if (isNaN(d)) return;
            display[c][i] = op.method ? d[op.method]() :
              op.options ? d.toLocaleDateString(op.locale, op.options) :
              d.toLocaleDateString(op.locale);
          });
          break;
        case "sub":
          eachCell(cols, (c, i, raw) => {
            if (op.small != null && isNum(raw) && raw !== 0 &&
                Math.abs(raw) < op.small) {
              display[c][i] = op.small_text ?? ("<" + op.small);
            } else if (op.zero != null && raw === 0) {
              display[c][i] = op.zero;
            } else if (op.missing != null && raw == null) {
              display[c][i] = op.missing;
            }
          });
          break;
        case "merge": {
          const mCols = op.columns;
          if (!mCols || mCols.length < 2) break;
          const target = mCols[0];
          for (let i = 0; i < nRow; i++) {
            const vals = mCols.map(c => display[c]?.[i] ?? "");
            display[target][i] = mergeVals(vals, op.pattern);
          }
          break;
        }
      }
    }
    return { display, nRow };
  }

  // --- Resolve structural spec fields from ops + data ---
  function resolveSpec(spec) {
    const data = spec.data || {},
          ops = spec.ops || [],
          colNames = Object.keys(data),
          nRow = colNames.length ? data[colNames[0]].length : 0,
          // Run fn on each op of the given type, in document order.
          onOp = (t, fn) => { for (const op of ops) if (op.type === t) fn(op); };

    // row_group: array → rowspan mode; string → separator-row mode
    let rowGroupSep = typeof spec.row_group === "string";
    const rowGroupCols = Array.isArray(spec.row_group) ? spec.row_group
          : (spec.row_group ? [spec.row_group] : []);
    if (!rowGroupSep && rowGroupCols.length === 1 && spec.auto_sep !== false &&
        !numCol(data[rowGroupCols[0]]) &&
        data[rowGroupCols[0]]?.some(v => (v + "").length > 20)) rowGroupSep = true;

    // Hidden columns: row_group, merge sources
    const hidden = new Set();
    for (const g of rowGroupCols) hidden.add(g);
    onOp("merge", op => {
      if (op.hide !== false && op.columns)
        op.columns.slice(1).forEach(c => hidden.add(c));
    });

    // Visible columns after hiding
    let visible = colNames.filter(c => !hidden.has(c));

    // Apply move
    onOp("move", op => {
      const toMove = (op.columns || []).filter(c => visible.includes(c));
      if (!toMove.length) return;
      const rest = visible.filter(c => !toMove.includes(c));
      if (op.after == null) {
        visible = [...toMove, ...rest];
      } else {
        const pos = rest.indexOf(op.after);
        if (pos >= 0) visible = [...rest.slice(0, pos + 1), ...toMove, ...rest.slice(pos + 1)];
      }
    });

    // Set arr[i] = val where i is the position of column c in `visible`.
    const setByCol = (arr, c, val) => { const i = visible.indexOf(c); if (i >= 0) arr[i] = val; };

    // Alignment: default from data type (number→right, else left), then overrides
    const align = visible.map(c => numCol(data[c]) ? "right" : "left");
    onOp("align", op => { for (const c of (op.columns || [])) setByCol(align, c, op.align); });

    // Column labels
    const autoLbl = s => spec.auto_label === false ? s : s.replace(/[._]/g, " ");
    const colLabels = visible.map(autoLbl);
    onOp("label", op => {
      for (const [c, lbl] of Object.entries(op.labels || {})) setByCol(colLabels, c, lbl);
    });

    // Column widths
    let colWidths = null;
    onOp("width", op => {
      if (!op.widths) return;
      if (!colWidths) colWidths = visible.map(() => "");
      for (const [c, w] of Object.entries(op.widths)) setByCol(colWidths, c, w);
    });

    // Indent
    const indent = new Array(nRow).fill(0);
    onOp("indent", op => {
      if (op.rows) for (const r of op.rows) indent[r - 1] = op.level ?? 1;
    });

    // Runs of equal consecutive values in col → [{ label, rows (1-based) }].
    // Used for both separator-row groups and rowspan span sizes.
    const runs = col => {
      const out = [];
      for (let i = 0; i < nRow;) {
        const label = str(col[i]), rows = [i + 1];
        while (++i < nRow && str(col[i]) === label) rows.push(i + 1);
        out.push({ label, rows });
      }
      return out;
    };

    // Row groups from data column (separator-row mode only)
    let groups = [];
    if (rowGroupSep && rowGroupCols.length && data[rowGroupCols[0]])
      groups.push(...runs(data[rowGroupCols[0]]));
    // Manual groups
    onOp("row_group", op => groups.push({ label: op.label, rows: op.rows }));
    // Group ordering
    onOp("group_order", op => {
      if (!groups.length) return;
      const ordered = [], rest = [];
      for (const lbl of (op.order || [])) {
        const g = groups.find(x => x.label === lbl);
        if (g) ordered.push(g);
      }
      for (const g of groups) if (!ordered.includes(g)) rest.push(g);
      groups = [...ordered, ...rest];
    });

    // Rowspan groups: compute span sizes for each group column
    const rowSpans = [];
    if (!rowGroupSep && rowGroupCols.length) {
      for (const gc of rowGroupCols) {
        const spans = new Array(nRow).fill(0);
        for (const r of runs(data[gc] || [])) spans[r.rows[0] - 1] = r.rows.length;
        // Resolve display label from label ops (last wins)
        let hdr = gc;
        onOp("label", op => { if (op.labels?.[gc] != null) hdr = op.labels[gc]; });
        rowSpans.push({ col: gc, label: hdr, spans });
      }
    }

    // Styles: the style ops are consumed directly (css/class/columns/rows/test).
    const styles = ops.filter(op => op.type === "style");

    // Auto-spanners: split column names on separator, group contiguous prefixes
    let spanners = spec.spanners || [];
    if (spec.auto_span) {
      const sep = typeof spec.auto_span === "string" ? new RegExp(spec.auto_span) : /[._]/;
      spanners = [...spanners];
      for (let i = 0; i < visible.length;) {
        const m = visible[i].match(sep);
        if (!m) { i++; continue; }
        const prefix = visible[i].slice(0, m.index);
        let j = i + 1;
        while (j < visible.length) {
          const m2 = visible[j].match(sep);
          if (!m2 || visible[j].slice(0, m2.index) !== prefix) break;
          j++;
        }
        if (j - i >= 2) {
          spanners.push({ label: prefix, columns: visible.slice(i, j) });
          for (let k = i; k < j; k++) {
            const mk = visible[k].match(sep);
            if (mk) colLabels[k] = autoLbl(visible[k].slice(mk.index + mk[0].length));
          }
        }
        i = j;
      }
    }

    return {
      visible, align, colLabels, colWidths, indent,
      groups, rowSpans, styles, spanners,
      footnotes: spec.footnotes || [],
      notes: spec.notes || [],
      header: spec.header || {},
      nRow
    };
  }

  // Footnotes: dedup by text, assign 1..N in first-seen order.
  function indexFootnotes(fns) {
    const order = [], idx = {};
    fns.forEach(f => {
      if (idx[f.text] == null) { idx[f.text] = order.length + 1; order.push(f.text); }
    });
    return { order, idx };
  }

  // Find the footnote index for a (type, value) location; 0 if none.
  // `val` is a scalar: the title group, a single column, a spanner label,
  // or a row-group label, depending on `type`.
  const matcher = (fns, idx) => (type, val) => {
    for (const f of fns) {
      const loc = f.location;
      if (loc.type !== type) continue;
      const hit = idx[f.text];
      if (type === "title" && loc.group === val) return hit;
      if (type === "column_labels" && loc.columns.includes(val)) return hit;
      if (type === "column_spanners" && loc.spanners.includes(val)) return hit;
      if (type === "row_groups") {
        if (loc.match === "all") return hit;
        if (loc.match === "exact" && loc.values.includes(val)) return hit;
        if (loc.match === "starts_with" && val.startsWith(loc.value)) return hit;
      }
    }
    return 0;
  };

  function sortByGroups(spec) {
    if (spec.sort === false || !spec.row_group) return;
    const data = spec.data || {},
          cols = [].concat(spec.row_group),
          colNames = Object.keys(data),
          nRow = colNames.length ? data[colNames[0]].length : 0;
    if (!nRow) return;
    const idx = Array.from({length: nRow}, (_, i) => i);
    idx.sort((a, b) => {
      for (const c of cols) {
        const va = data[c]?.[a], vb = data[c]?.[b];
        if (va == vb) continue;
        if (va == null) return 1;
        if (vb == null) return -1;
        if (va < vb) return -1;
        if (va > vb) return 1;
      }
      return 0;
    });
    for (const c of colNames) data[c] = idx.map(i => data[c][i]);
    // Remap 1-based row indices in ops to reflect the new order
    const newPos = new Array(nRow);
    for (let i = 0; i < nRow; i++) newPos[idx[i]] = i + 1;
    for (const op of (spec.ops || [])) {
      if (op.rows) op.rows = op.rows.map(r => newPos[r - 1]);
    }
  }

  function buildHtml(spec) {
    sortByGroups(spec);
    const data = spec.data || {},
          { display, nRow } = applyOps(spec),
          { visible: cols, align, colLabels, colWidths, indent,
            groups, rowSpans, styles, spanners, footnotes: fns, notes, header: hdr } = resolveSpec(spec),
          reg = indexFootnotes(fns),
          fIdx = matcher(fns, reg.idx),
          nGrp = rowSpans.length,
          nCol = cols.length + nGrp,
          out = [`<table class="lt-table">`];
    const mark = (type, val) => { const i = fIdx(type, val); return i ? sup(i) : ""; };
    const cell = (c, r) => display[c]?.[r - 1] ?? "";
    // ` name="val"` for a truthy val, else "" — for optional HTML attributes.
    const attr = (n, v) => v ? ` ${n}="${v}"` : "";
    // Plain class names per column (alignment + leading-indent), "" if none.
    const colCls = cols.map((_, i) => (
      (i === 0 && groups.length ? "lt-indent " : "") +
      ({right: "al-r", center: "al-c"}[align[i]] || "")
    ).trimEnd());

    // Visit each cell at (rows × cols) that exists in the table, calling
    // fn(key, c, r). rows null = all rows; cols falls back to all columns.
    const eachLoc = (rows, locCols, fn) => {
      for (let r = 1; r <= nRow; r++) {
        if (rows && !rows.includes(r)) continue;
        for (const c of (locCols || cols)) {
          const ci = cols.indexOf(c);
          if (ci >= 0) fn(`${r},${ci}`, c, r);
        }
      }
    };

    // Style map: accumulate css (";") and class (" ") per cell.
    const styleMap = {}, classMap = {};
    const addTo = (map, key, val, sep) => { map[key] = map[key] ? map[key] + sep + val : val; };
    for (const s of styles)
      eachLoc(s.rows, s.columns ? [].concat(s.columns) : null, (key, c, r) => {
        if (s.test && !s.test(data[c][r - 1])) return;
        if (s.css) addTo(styleMap, key, s.css, ";");
        if (s.class) addTo(classMap, key, s.class, " ");
      });

    // <colgroup>
    if (colWidths && colWidths.some(w => w)) {
      out.push(`<colgroup>`);
      for (let i = 0; i < cols.length; i++)
        out.push(`<col${attr("style", colWidths[i] ? "width:" + colWidths[i] : "")}>`);
      out.push(`</colgroup>`);
    }

    // <caption>: title + subtitle, each a <div> rendered only when non-empty.
    const caption = [["title", "lt-title"], ["subtitle", "lt-subtitle"]]
      .filter(([g]) => hdr[g] != null && hdr[g] !== "")
      .map(([g, cls]) => `<div class="${cls}">${esc(hdr[g])}${mark("title", g)}</div>`);
    if (caption.length)
      out.push(`<caption class="lt-caption">`, ...caption, `</caption>`);

    // <thead>
    out.push(`<thead>`);
    if (spanners.length) {
      const emptyTh = `<th class="lt-spanner-empty"></th>`;
      out.push(`<tr class="lt-spanner-row">`);
      out.push(emptyTh.repeat(nGrp));
      for (let k = 0; k < cols.length;) {
        const sp = spanners.find(s => s.columns[0] === cols[k]);
        if (sp) {
          out.push(`<th colspan="${sp.columns.length}" scope="colgroup" class="lt-spanner">${esc(sp.label)}${mark("column_spanners", sp.label)}</th>`);
          k += sp.columns.length;
        } else {
          out.push(emptyTh);
          k++;
        }
      }
      out.push(`</tr>`);
    }
    out.push(`<tr>`);
    for (const rs of rowSpans) out.push(`<th scope="col">${esc(rs.label)}</th>`);
    for (let i = 0; i < cols.length; i++)
      out.push(`<th scope="col"${attr("class", colCls[i])}>${esc(colLabels[i])}${mark("column_labels", cols[i])}</th>`);
    out.push(`</tr></thead>`);

    // Body footnote markers
    const bodyMarks = {};
    for (const f of fns) {
      if (f.location.type !== "body") continue;
      const fi = reg.idx[f.text];
      eachLoc(f.location.rows, f.location.columns || [], key => { bodyMarks[key] = fi; });
    }

    // Row rendering
    const pushRow = r => {
      out.push(`<tr>`);
      // Rowspan group cells
      for (const rs of rowSpans) {
        const span = rs.spans[r - 1];
        if (span > 0) out.push(`<th scope="row" class="lt-row-group"${attr("rowspan", span > 1 ? span : 0)}>${esc(cell(rs.col, r))}</th>`);
      }
      const ind = indent[r - 1] || 0;
      for (let ci = 0; ci < cols.length; ci++) {
        const k = `${r},${ci}`,
              m = bodyMarks[k], cc = classMap[k],
              cls = [colCls[ci], cc].filter(Boolean).join(" ");
        let s = styleMap[k] || "";
        if (ci === 0 && ind) s = (s ? s + ";" : "") + `padding-left:${ind + 1}em`;
        out.push(`<td${attr("class", cls)}${attr("style", s)}>${esc(cell(cols[ci], r))}${m ? sup(m) : ""}</td>`);
      }
      out.push(`</tr>`);
    };

    // <tbody>
    out.push(`<tbody>`);
    if (groups.length) {
      const seen = {};
      for (const g of groups) {
        out.push(`<tr class="lt-row-group"><th colspan="${nCol}" scope="colgroup">${esc(g.label)}${mark("row_groups", g.label)}</th></tr>`);
        for (const r of g.rows) { seen[r] = 1; pushRow(r); }
      }
      for (let r = 1; r <= nRow; r++) if (!seen[r]) pushRow(r);
    } else {
      for (let r = 1; r <= nRow; r++) pushRow(r);
    }
    out.push(`</tbody>`);

    // <tfoot>: footnotes then source notes, each a full-width row.
    if (reg.order.length || notes.length) {
      const footRow = (cls, html) => `<tr class="${cls}"><td colspan="${nCol}">${html}</td></tr>`;
      out.push(`<tfoot class="lt-footer">`);
      reg.order.forEach((txt, i) => out.push(footRow("lt-footnote", `${sup(i + 1)} ${esc(txt)}`)));
      for (const n of notes) out.push(footRow("lt-source-note", esc(n)));
      out.push(`</tfoot>`);
    }

    out.push(`</table>`);
    return out.join("");
  }

  const mount = (s, spec) => s.insertAdjacentHTML("afterend", buildHtml(spec));
  // q.push renders immediately; replay any entries queued before we loaded.
  const q = { push: e => mount(e.s, e.d) };
  (root.LT?.q || []).forEach(q.push);
  root.LT = { build: spec => mount(document.currentScript, spec), buildHtml, q };
})(window);
</script>
</div>
</div>
<p>It’s important to note that this is not an apples to apples comparison.</p>
<p>The bottleneck for dplyr/tidyr, pandas, tidytable, and polars is the unnesting operation, but there is no unnesting operation for DataFrames.jl<sup>9</sup>, data.table, and the naive Python solution.</p>
<div class="no-row-height column-margin column-container"><div id="fn9"><p><sup>9</sup>&nbsp;DataFrames.jl seems to do this automatically for the named tuple?</p></div></div><p>Still, it’s no surprise that Julia wins by a huge margin. It’s one of the reasons why I have moved entirely from Python to Julia.</p>
<p>However, my take is that we have to make the choice between coding time and computing time. Using the unnesting operation is quite convenient, but sacrifices speed. For example, I’m sure that polars can be made much faster if the code were more idiomatic to polars.</p>
<p>Do I want to sacrifice the readable and simple tidyverse codes for faster runtime, either by using a different package, writing in the naive way, or using Julia<sup>10</sup>?</p>
<div class="no-row-height column-margin column-container"><div id="fn10"><p><sup>10</sup>&nbsp;Perhaps one day <a href="https://tidierorg.github.io/Tidier.jl/stable/">Tidier.jl</a> will reach the same level as the tidyverse and all will be good.</p></div></div><p>For my purposes, probably not. My bottlenecks tend to be the computation itself (i.e., <code>hugeQuantumSys</code>) rather than the one time data wrangling anyway. For now, a little improvements using drop-in replacements for tidyverse is good enough.</p>
</section>
<section id="interops-with-python-and-c" class="level2 page-columns page-full">
<h2 class="anchored" data-anchor-id="interops-with-python-and-c">Interops with Python and C++</h2>
<p>But R don’t have the myriads of packages and libraries you have in Python you say?</p>
<p>I hear you. In fact, I’m a little annoyed that I can’t find a <a href="https://qutip.org/">QuTiP</a>, <a href="https://qutip.org/QuantumToolbox.jl/stable/">QuantumToolbox.jl</a>, or <a href="https://qojulia.org/">QuantumOptics.jl</a> equivalent in R<sup>11</sup>.</p>
<div class="no-row-height column-margin column-container"><div id="fn11"><p><sup>11</sup>&nbsp;There’s so many quantum <strong>computing</strong> packages though, which are just glorified <a href="https://en.wikipedia.org/wiki/Domain-specific_language">DSL</a> for matrix multiplications.</p></div></div><p>But no worries, we can simply do this:</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb18" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb18-1">reticulate<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">::</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">use_virtualenv</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"~/.local/venv/"</span>)</span>
<span id="cb18-2">qt <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> reticulate<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">::</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">import</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"qutip"</span>)</span>
<span id="cb18-3"></span>
<span id="cb18-4">J <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span></span>
<span id="cb18-5">t <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span></span>
<span id="cb18-6"></span>
<span id="cb18-7">Hint <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> J <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span></span>
<span id="cb18-8">    qt<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">$</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">tensor</span>(qt<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">$</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">destroy</span>(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>L), qt<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">$</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">create</span>(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>L)) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb18-9">    qt<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">$</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">tensor</span>(qt<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">$</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">create</span>(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>L), qt<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">$</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">destroy</span>(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>L))</span>
<span id="cb18-10"></span>
<span id="cb18-11">U <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> (<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>i <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> Hint <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> t)<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">$</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">expm</span>()</span>
<span id="cb18-12"></span>
<span id="cb18-13">rhoIn <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> qt<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">$</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">rand_dm</span>(<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">c</span>(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>L, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>L))</span>
<span id="cb18-14">rhoOut <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> U <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> rhoIn <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> U<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">$</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">dag</span>()</span>
<span id="cb18-15"></span>
<span id="cb18-16"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">print</span>(rhoOut)</span></code></pre></div></div>
<pre><code>Quantum object: dims=[[2, 2], [2, 2]], shape=(4, 4), type='oper', dtype=Dense, isherm=True
Qobj data =
[[ 0.23766069+0.00000000e+00j -0.12235266-1.04414855e-01j
   0.1342473 -3.25719171e-02j  0.01416946+8.07667926e-02j]
 [-0.12235266+1.04414855e-01j  0.36023742+0.00000000e+00j
  -0.09647204+1.98926694e-01j -0.24404032-3.03698913e-02j]
 [ 0.1342473 +3.25719171e-02j -0.09647204-1.98926694e-01j
   0.15602805-6.93889390e-18j  0.0481154 +1.36967243e-01j]
 [ 0.01416946-8.07667926e-02j -0.24404032+3.03698913e-02j
   0.0481154 -1.36967243e-01j  0.24607384+0.00000000e+00j]]
 signature: (other: 'Qobj') -&gt; 'Qobj'</code></pre>

<div class="no-row-height column-margin column-container"><div class="margin-aside">
<p><br>
</p>
<p>In case you are wondering what we are doing here. <img src="https://latex.codecogs.com/png.latex?%0AH_%5Ctext%7Bint%7D%20=%20J(%5Csigma_-%20%5Csigma_+%20+%20%5Csigma_+%20%5Csigma_-),%0A"> <img src="https://latex.codecogs.com/png.latex?%0AU%20=%20e%5E%7B-iH_%5Ctext%7Bint%7D%20t%7D,%0A"> <img src="https://latex.codecogs.com/png.latex?%0A%5Crho_%5Ctext%7Bout%7D%20=%20U%20%5Crho_%5Ctext%7Bin%7D%20U%5E%5Cdagger,%0A"> where <img src="https://latex.codecogs.com/png.latex?%5Crho_%5Ctext%7Bin%7D"> is a randomly generated density matrix.</p>
</div></div><p>And just like that, we are using QuTiP in R.</p>
<p>Thanks to the <a href="https://rstudio.github.io/reticulate/">reticulate</a> package, I activated my Python virtual environment, and imported QuTiP to <code>qt</code>, which allows us to use it like in Python, just with some small syntax changes.</p>
<p>We can continue to use Python packages but with the goodies of tidyverse and ggplot2!</p>
<p>And if R and Python are not fast enough for your computationally expensive codes, <a href="https://www.rcpp.org/">Rcpp</a> allows the calling of C++ code easily.</p>
<p>For example, I can use <a href="https://github.com/RcppCore/RcppEigen">RcppEigen</a> (or <a href="https://github.com/RcppCore/RcppArmadillo">RcppArmadillo</a>) with <a href="https://github.com/softwareQinc/qpp">Quantum++</a> to write a <code>compression.cpp</code> file which exports the C++ function <code>compressionGate</code> that I can call in <code>main.R</code>.</p>
<div class="tabset-margin-container"></div><div class="panel-tabset">
<ul class="nav nav-tabs"><li class="nav-item"><a class="nav-link active" id="tabset-5-1-tab" data-bs-toggle="tab" data-bs-target="#tabset-5-1" aria-controls="tabset-5-1" aria-selected="true" href="">main.R</a></li><li class="nav-item"><a class="nav-link" id="tabset-5-2-tab" data-bs-toggle="tab" data-bs-target="#tabset-5-2" aria-controls="tabset-5-2" aria-selected="false" href="">compression.cpp</a></li></ul>
<div class="tab-content">
<div id="tabset-5-1" class="tab-pane active" aria-labelledby="tabset-5-1-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb20" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb20-1"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">library</span>(RcppEigen)</span>
<span id="cb20-2"></span>
<span id="cb20-3">reticulate<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">::</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">use_virtualenv</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"~/.local/venv/"</span>)</span>
<span id="cb20-4">qt <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> reticulate<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">::</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">import</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"qutip"</span>)</span>
<span id="cb20-5"></span>
<span id="cb20-6"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">Sys.setenv</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">PKG_CXXFLAGS =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"-I/usr/local/include/qpp"</span>)</span>
<span id="cb20-7"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">sourceCpp</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"compression.cpp"</span>)</span>
<span id="cb20-8"></span>
<span id="cb20-9">rhoIn <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> qt<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">$</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">rand_dm</span>(<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">c</span>(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>L, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>L, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>L))<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">$</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">full</span>()</span>
<span id="cb20-10"></span>
<span id="cb20-11">rhoOut <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">compressionGate</span>(rhoIn)</span></code></pre></div></div>
</div>
<div id="tabset-5-2" class="tab-pane" aria-labelledby="tabset-5-2-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb21" style="background: #f1f3f5;"><pre class="sourceCode cpp code-with-copy"><code class="sourceCode cpp"><span id="cb21-1"><span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">#include </span><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">&lt;RcppEigen.h&gt;</span></span>
<span id="cb21-2"></span>
<span id="cb21-3"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// [[Rcpp::depends(RcppEigen)]]</span></span>
<span id="cb21-4"></span>
<span id="cb21-5"><span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">#include </span><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">&lt;cmath&gt;</span></span>
<span id="cb21-6"><span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">#include </span><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">&lt;qpp/qpp.hpp&gt;</span></span>
<span id="cb21-7"></span>
<span id="cb21-8"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">using</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">namespace</span> qpp<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb21-9"></span>
<span id="cb21-10"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">// [[Rcpp:export]]</span></span>
<span id="cb21-11">cmat compressionGate<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>cmat rhoIn<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">{</span></span>
<span id="cb21-12">  cmat compress1 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> kron<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>st<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>pz0<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> gt<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>Id2<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> gt<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>Id2<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> kron<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>st<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>pz1<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> gt<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>X<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> gt<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>Id2<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">);</span></span>
<span id="cb21-13">  compress1 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*=</span> kron<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>st<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>pz0<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> gt<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>Id2<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> gt<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>Id2<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> kron<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>st<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>pz1<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> gt<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>Id2<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> gt<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>X<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">);</span></span>
<span id="cb21-14">  cmat compress2 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> kron<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>gt<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>Id2<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> st<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>pz0<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> st<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>pz0<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> kron<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>gt<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>Id2<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> st<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>pz0<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> st<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>pz1<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb21-15">                   kron<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>gt<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>Id2<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> st<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>pz1<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> st<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>pz0<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> kron<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>gt<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>X<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> st<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>pz1<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> st<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>pz1<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">);</span></span>
<span id="cb21-16">  cmat compress <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> compress1 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> compress2 <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> compress1<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">;</span></span>
<span id="cb21-17"></span>
<span id="cb21-18">  <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">return</span> compress <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> rhoIn <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> adjoint<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>compress<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">);</span></span>
<span id="cb21-19"><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">}</span></span></code></pre></div></div>
</div>
</div>
</div>

<div class="no-row-height column-margin column-container"><div class="margin-aside">
<p>Of course, this is just a toy example, and in actual use cases you will be writing the computationally expensive parts in C++ instead of simple matrix multiplications.</p>
</div></div><p>Here, I generated a random three qubits state with Python’s QuTiP and apply the C++-written <code>compressionGate</code> on it.</p>
<p>Note that it’s also possible to interoperate with <a href="https://github.com/qinwf/awesome-R#language-api">other languages</a>, but as far as I’m aware, reticulate and Rcpp are the most established.</p>
</section>
</section>
<section id="but-llm" class="level1 page-columns page-full">
<h1>But LLM?</h1>
<p>There are some growing sentiments that it no longer matters what language or package one uses, as one can simply ask an LLM to write the code for you.</p>
<p>Here, I’ll make two claims that I believe applies to the current state of LLM at the time of writing (June 2026), that leads to corollaries on why I think the language still matters.</p>
<ol type="1">
<li><p>Current LLMs are not perfect, and can still generate bug-ridden codes.</p></li>
<li><p>Different from software development, in scientific computing and data analysis, bugs can be invisible by simply manifesting as plots or values that are incorrect.</p></li>
</ol>
<p>The corollary to these is that</p>
<ol start="3" type="1">
<li>One has to check and audit the LLM-generated code for any serious work.</li>
</ol>
<p>I mean, would you submit results generated by LLM-written codes for publication without checking it? Or would you advise your company or clients using the same results without ensuring that the LLM-written codes are correct?</p>
<p>Considering these, my opinion is that</p>
<ol start="4" type="1">
<li>The tidyverse syntax and way of doing things is easier and faster to check and audit.</li>
</ol>
<p>On top of its simplicity, the tidyverse has a somewhat declarative syntax especially when used with higher-order functions (purrr) which makes it even easier to check for correctness<sup>12</sup>.</p>
<div class="no-row-height column-margin column-container"><div id="fn12"><p><sup>12</sup>&nbsp;I’ve talked about the same thing in a <a href="../posts/haskell-numerical.html">Haskell post</a> regarding how it is easier to check for correctness in a declarative and functional approach rather than an imperative one.</p></div></div><p>I’ll go even further to say that the declarative approach means that using the tidyverse actually saves you more time than using an LLM, especially considering that LLMs like to generate overly complex codes rather than keeping it simple.</p>
<p>Imagine telling an LLM that<sup>13</sup>:</p>
<div class="no-row-height column-margin column-container"><div id="fn13"><p><sup>13</sup>&nbsp;We use pandas as an example here, but the point probably still stands for non-tidyverse style codes.</p></div></div><blockquote class="blockquote">
<p>“write me a pandas code for a dataframe df that keeps only y1 less than 2, create a new column where z is the sum of y1, y2, y3, then plot z against x1 where there is two lines, one for z &gt;= 0 and one for z &lt; 0”</p>
</blockquote>
<p>wait for it to generate, copy and paste it, run it and hope that it didn’t hallucinate some functions, and then check that it is actually doing what you want, when you could have type less characters if you do it yourself.</p>
<div class="tabset-margin-container"></div><div class="panel-tabset">
<ul class="nav nav-tabs"><li class="nav-item"><a class="nav-link active" id="tabset-6-1-tab" data-bs-toggle="tab" data-bs-target="#tabset-6-1" aria-controls="tabset-6-1" aria-selected="true" href="">What you could have just written</a></li><li class="nav-item"><a class="nav-link" id="tabset-6-2-tab" data-bs-toggle="tab" data-bs-target="#tabset-6-2" aria-controls="tabset-6-2" aria-selected="false" href="">GPT-5 mini (pandas)</a></li><li class="nav-item"><a class="nav-link" id="tabset-6-3-tab" data-bs-toggle="tab" data-bs-target="#tabset-6-3" aria-controls="tabset-6-3" aria-selected="false" href="">GPT-5 mini (tidyverse)</a></li><li class="nav-item"><a class="nav-link" id="tabset-6-4-tab" data-bs-toggle="tab" data-bs-target="#tabset-6-4" aria-controls="tabset-6-4" aria-selected="false" href="">Claude Haiku 4.5 (pandas)</a></li><li class="nav-item"><a class="nav-link" id="tabset-6-5-tab" data-bs-toggle="tab" data-bs-target="#tabset-6-5" aria-controls="tabset-6-5" aria-selected="false" href="">Claude Haiku 4.5 (tidyverse)</a></li><li class="nav-item"><a class="nav-link" id="tabset-6-6-tab" data-bs-toggle="tab" data-bs-target="#tabset-6-6" aria-controls="tabset-6-6" aria-selected="false" href="">Llama 4 Scout (pandas)</a></li><li class="nav-item"><a class="nav-link" id="tabset-6-7-tab" data-bs-toggle="tab" data-bs-target="#tabset-6-7" aria-controls="tabset-6-7" aria-selected="false" href="">Llama 4 Scout (tidyverse)</a></li><li class="nav-item"><a class="nav-link" id="tabset-6-8-tab" data-bs-toggle="tab" data-bs-target="#tabset-6-8" aria-controls="tabset-6-8" aria-selected="false" href="">Mistral Small 4 (pandas)</a></li><li class="nav-item"><a class="nav-link" id="tabset-6-9-tab" data-bs-toggle="tab" data-bs-target="#tabset-6-9" aria-controls="tabset-6-9" aria-selected="false" href="">Mistral Small 4 (tidyverse)</a></li><li class="nav-item"><a class="nav-link" id="tabset-6-10-tab" data-bs-toggle="tab" data-bs-target="#tabset-6-10" aria-controls="tabset-6-10" aria-selected="false" href="">gpt-oss 120B (pandas)</a></li><li class="nav-item"><a class="nav-link" id="tabset-6-11-tab" data-bs-toggle="tab" data-bs-target="#tabset-6-11" aria-controls="tabset-6-11" aria-selected="false" href="">gpt-oss 120B (tidyverse)</a></li><li class="nav-item"><a class="nav-link" id="tabset-6-12-tab" data-bs-toggle="tab" data-bs-target="#tabset-6-12" aria-controls="tabset-6-12" aria-selected="false" href="">Proton Lumo (pandas)</a></li><li class="nav-item"><a class="nav-link" id="tabset-6-13-tab" data-bs-toggle="tab" data-bs-target="#tabset-6-13" aria-controls="tabset-6-13" aria-selected="false" href="">Proton Lumo (tidyverse)</a></li></ul>
<div class="tab-content">
<div id="tabset-6-1" class="tab-pane active" aria-labelledby="tabset-6-1-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb22" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb22-1">df <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|&gt;</span></span>
<span id="cb22-2">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">filter</span>(y1 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|&gt;</span></span>
<span id="cb22-3">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">mutate</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">z =</span> y1 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> y2 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> y3) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|&gt;</span></span>
<span id="cb22-4">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">ggplot</span>(<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">aes</span>(x1, z, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">color =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">factor</span>(z <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;=</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>))) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb22-5">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">geom_line</span>()</span></code></pre></div></div>
</div>
<div id="tabset-6-2" class="tab-pane" aria-labelledby="tabset-6-2-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb23" style="background: #f1f3f5;"><pre class="sourceCode py code-with-copy"><code class="sourceCode python"><span id="cb23-1"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">import</span> pandas <span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">as</span> pd</span>
<span id="cb23-2"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">import</span> matplotlib.pyplot <span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">as</span> plt</span>
<span id="cb23-3"></span>
<span id="cb23-4"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># example: assume df already exists</span></span>
<span id="cb23-5"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Step 1: keep only rows where y1 &lt; 2</span></span>
<span id="cb23-6">df <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> df[df[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'y1'</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>].copy()</span>
<span id="cb23-7"></span>
<span id="cb23-8"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Step 2: create new column z = y1 + y2 + y3</span></span>
<span id="cb23-9">df[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z'</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> df[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'y1'</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> df[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'y2'</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> df[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'y3'</span>]</span>
<span id="cb23-10"></span>
<span id="cb23-11"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Step 3: split into two series for plotting</span></span>
<span id="cb23-12">df_pos <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> df[df[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z'</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;=</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>]</span>
<span id="cb23-13">df_neg <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> df[df[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z'</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>]</span>
<span id="cb23-14"></span>
<span id="cb23-15"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Step 4: plot z vs x1 with two lines</span></span>
<span id="cb23-16">plt.figure(figsize<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span>(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">8</span>,<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">5</span>))</span>
<span id="cb23-17">plt.plot(df_pos[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'x1'</span>], df_pos[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z'</span>], label<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z &gt;= 0'</span>, color<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'tab:blue'</span>, marker<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'o'</span>, linestyle<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'-'</span>)</span>
<span id="cb23-18">plt.plot(df_neg[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'x1'</span>], df_neg[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z'</span>], label<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z &lt; 0'</span>, color<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'tab:red'</span>, marker<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'o'</span>, linestyle<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'-'</span>)</span>
<span id="cb23-19">plt.xlabel(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'x1'</span>)</span>
<span id="cb23-20">plt.ylabel(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z'</span>)</span>
<span id="cb23-21">plt.title(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z vs x1 (separate lines for z &gt;= 0 and z &lt; 0)'</span>)</span>
<span id="cb23-22">plt.legend()</span>
<span id="cb23-23">plt.grid(<span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">True</span>)</span>
<span id="cb23-24">plt.tight_layout()</span>
<span id="cb23-25">plt.show()</span></code></pre></div></div>
</div>
<div id="tabset-6-3" class="tab-pane" aria-labelledby="tabset-6-3-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb24" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb24-1"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">library</span>(tidyverse)</span>
<span id="cb24-2"></span>
<span id="cb24-3"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># assume df exists with columns x1, y1, y2, y3</span></span>
<span id="cb24-4">df_clean <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> df <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb24-5">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">filter</span>(y1 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span>                         <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># keep only y1 &lt; 2</span></span>
<span id="cb24-6">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">mutate</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">z =</span> y1 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> y2 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> y3,                   <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># new z column</span></span>
<span id="cb24-7">         <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">z_sign =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">if_else</span>(z <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;=</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"z &gt;= 0"</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"z &lt; 0"</span>))  <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># grouping column for lines</span></span>
<span id="cb24-8"></span>
<span id="cb24-9"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Plot: two lines (one for z &gt;= 0, one for z &lt; 0)</span></span>
<span id="cb24-10"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">ggplot</span>(df_clean, <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">aes</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">x =</span> x1, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">y =</span> z, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">color =</span> z_sign)) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb24-11">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">geom_line</span>() <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb24-12">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">geom_point</span>() <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span>                             <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># optional: points</span></span>
<span id="cb24-13">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">scale_color_manual</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">values =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">c</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"z &gt;= 0"</span> <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"steelblue"</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"z &lt; 0"</span> <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"firebrick"</span>)) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb24-14">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">labs</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">x =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"x1"</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">y =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"z (y1+y2+y3)"</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">color =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Group"</span>) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb24-15">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">theme_minimal</span>()</span></code></pre></div></div>
</div>
<div id="tabset-6-4" class="tab-pane" aria-labelledby="tabset-6-4-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb25" style="background: #f1f3f5;"><pre class="sourceCode py code-with-copy"><code class="sourceCode python"><span id="cb25-1"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">import</span> pandas <span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">as</span> pd</span>
<span id="cb25-2"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">import</span> matplotlib.pyplot <span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">as</span> plt</span>
<span id="cb25-3"></span>
<span id="cb25-4"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Filter dataframe to keep only rows where y1 &lt; 2</span></span>
<span id="cb25-5">df <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> df[df[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'y1'</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>]</span>
<span id="cb25-6"></span>
<span id="cb25-7"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Create new column z as the sum of y1, y2, y3</span></span>
<span id="cb25-8">df[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z'</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> df[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'y1'</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> df[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'y2'</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> df[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'y3'</span>]</span>
<span id="cb25-9"></span>
<span id="cb25-10"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Create the plot</span></span>
<span id="cb25-11">plt.figure(figsize<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span>(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">10</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">6</span>))</span>
<span id="cb25-12"></span>
<span id="cb25-13"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Plot line for z &gt;= 0</span></span>
<span id="cb25-14">df_positive <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> df[df[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z'</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;=</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>]</span>
<span id="cb25-15">plt.plot(df_positive[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'x1'</span>], df_positive[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z'</span>], label<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z &gt;= 0'</span>, marker<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'o'</span>)</span>
<span id="cb25-16"></span>
<span id="cb25-17"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Plot line for z &lt; 0</span></span>
<span id="cb25-18">df_negative <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> df[df[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z'</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>]</span>
<span id="cb25-19">plt.plot(df_negative[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'x1'</span>], df_negative[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z'</span>], label<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z &lt; 0'</span>, marker<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'o'</span>)</span>
<span id="cb25-20"></span>
<span id="cb25-21">plt.xlabel(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'x1'</span>)</span>
<span id="cb25-22">plt.ylabel(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z'</span>)</span>
<span id="cb25-23">plt.legend()</span>
<span id="cb25-24">plt.grid(<span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">True</span>)</span>
<span id="cb25-25">plt.show()</span></code></pre></div></div>
</div>
<div id="tabset-6-5" class="tab-pane" aria-labelledby="tabset-6-5-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb26" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb26-1"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">library</span>(tidyverse)</span>
<span id="cb26-2"></span>
<span id="cb26-3">df <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb26-4">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">filter</span>(y1 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb26-5">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">mutate</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">z =</span> y1 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> y2 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> y3) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb26-6">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">ggplot</span>(<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">aes</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">x =</span> x1, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">y =</span> z, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">color =</span> z <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;=</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>)) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb26-7">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">geom_line</span>() <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb26-8">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">scale_color_manual</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">values =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">c</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"FALSE"</span> <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"red"</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"TRUE"</span> <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"blue"</span>),</span>
<span id="cb26-9">                     <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">labels =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">c</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"FALSE"</span> <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"z &lt; 0"</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"TRUE"</span> <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"z &gt;= 0"</span>)) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb26-10">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">labs</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">color =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Condition"</span>,</span>
<span id="cb26-11">       <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">x =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"x1"</span>,</span>
<span id="cb26-12">       <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">y =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"z"</span>) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb26-13">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">theme_minimal</span>()</span></code></pre></div></div>
</div>
<div id="tabset-6-6" class="tab-pane" aria-labelledby="tabset-6-6-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb27" style="background: #f1f3f5;"><pre class="sourceCode py code-with-copy"><code class="sourceCode python"><span id="cb27-1"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">import</span> pandas <span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">as</span> pd</span>
<span id="cb27-2"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">import</span> matplotlib.pyplot <span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">as</span> plt</span>
<span id="cb27-3"></span>
<span id="cb27-4"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Assuming df is your DataFrame</span></span>
<span id="cb27-5">df <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> df[df[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'y1'</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>]  <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Keep only rows where y1 is less than 2</span></span>
<span id="cb27-6"></span>
<span id="cb27-7"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Create a new column 'z' which is the sum of y1, y2, y3</span></span>
<span id="cb27-8">df[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z'</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> df[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'y1'</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> df[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'y2'</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> df[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'y3'</span>]</span>
<span id="cb27-9"></span>
<span id="cb27-10"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Plot 'z' against 'x1'</span></span>
<span id="cb27-11">plt.figure(figsize<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span>(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">10</span>,<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">6</span>))</span>
<span id="cb27-12">df_pos <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> df[df[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z'</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;=</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>]</span>
<span id="cb27-13">df_neg <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> df[df[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z'</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>]</span>
<span id="cb27-14"></span>
<span id="cb27-15">plt.plot(df_pos[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'x1'</span>], df_pos[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z'</span>], label<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z &gt;= 0'</span>)</span>
<span id="cb27-16">plt.plot(df_neg[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'x1'</span>], df_neg[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z'</span>], label<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z &lt; 0'</span>)</span>
<span id="cb27-17"></span>
<span id="cb27-18">plt.xlabel(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'x1'</span>)</span>
<span id="cb27-19">plt.ylabel(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z'</span>)</span>
<span id="cb27-20">plt.title(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'Plot of z against x1'</span>)</span>
<span id="cb27-21">plt.legend()</span>
<span id="cb27-22">plt.grid(<span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">True</span>)</span>
<span id="cb27-23">plt.show()</span></code></pre></div></div>
</div>
<div id="tabset-6-7" class="tab-pane" aria-labelledby="tabset-6-7-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb28" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb28-1"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">library</span>(tidyverse)</span>
<span id="cb28-2"></span>
<span id="cb28-3">df <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb28-4">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">filter</span>(y1 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb28-5">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">mutate</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">z =</span> y1 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> y2 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> y3) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb28-6">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">ggplot</span>(<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">aes</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">x =</span> x1, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">y =</span> z, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">color =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">factor</span>(z <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;=</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>))) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb28-7">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">geom_line</span>() <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb28-8">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">labs</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">color =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"z &gt;= 0"</span>)</span></code></pre></div></div>
</div>
<div id="tabset-6-8" class="tab-pane" aria-labelledby="tabset-6-8-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb29" style="background: #f1f3f5;"><pre class="sourceCode py code-with-copy"><code class="sourceCode python"><span id="cb29-1"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">import</span> pandas <span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">as</span> pd</span>
<span id="cb29-2"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">import</span> matplotlib.pyplot <span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">as</span> plt</span>
<span id="cb29-3"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">import</span> numpy <span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">as</span> np</span>
<span id="cb29-4"></span>
<span id="cb29-5"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Sample data creation (replace this with your actual DataFrame)</span></span>
<span id="cb29-6">np.random.seed(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">42</span>)</span>
<span id="cb29-7">df <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> pd.DataFrame({</span>
<span id="cb29-8">    <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'x1'</span>: np.linspace(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">10</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">100</span>),</span>
<span id="cb29-9">    <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'y1'</span>: np.random.uniform(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">3</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">100</span>),</span>
<span id="cb29-10">    <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'y2'</span>: np.random.uniform(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">100</span>),</span>
<span id="cb29-11">    <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'y3'</span>: np.random.uniform(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">100</span>)</span>
<span id="cb29-12">})</span>
<span id="cb29-13"></span>
<span id="cb29-14"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Filter rows where y1 &lt; 2</span></span>
<span id="cb29-15">df_filtered <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> df[df[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'y1'</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>].copy()</span>
<span id="cb29-16"></span>
<span id="cb29-17"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Create new column z as sum of y1, y2, y3</span></span>
<span id="cb29-18">df_filtered[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z'</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> df_filtered[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'y1'</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> df_filtered[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'y2'</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> df_filtered[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'y3'</span>]</span>
<span id="cb29-19"></span>
<span id="cb29-20"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Create a categorical column for z &gt;= 0 and z &lt; 0</span></span>
<span id="cb29-21">df_filtered[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z_category'</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> np.where(df_filtered[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z'</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;=</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z &gt;= 0'</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z &lt; 0'</span>)</span>
<span id="cb29-22"></span>
<span id="cb29-23"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Plot z against x1 with two lines</span></span>
<span id="cb29-24">plt.figure(figsize<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span>(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">10</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">6</span>))</span>
<span id="cb29-25"><span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">for</span> category, group <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">in</span> df_filtered.groupby(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z_category'</span>):</span>
<span id="cb29-26">    plt.plot(group[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'x1'</span>], group[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z'</span>], label<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span>category)</span>
<span id="cb29-27"></span>
<span id="cb29-28">plt.xlabel(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'x1'</span>)</span>
<span id="cb29-29">plt.ylabel(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z'</span>)</span>
<span id="cb29-30">plt.title(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z vs x1 by z category'</span>)</span>
<span id="cb29-31">plt.legend()</span>
<span id="cb29-32">plt.grid(<span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">True</span>)</span>
<span id="cb29-33">plt.show()</span></code></pre></div></div>
</div>
<div id="tabset-6-9" class="tab-pane" aria-labelledby="tabset-6-9-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb30" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb30-1"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">library</span>(tidyverse)</span>
<span id="cb30-2"></span>
<span id="cb30-3"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Assuming df is your dataframe</span></span>
<span id="cb30-4">df_filtered <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> df <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb30-5">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">filter</span>(y1 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb30-6">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">mutate</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">z =</span> y1 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> y2 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> y3) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb30-7">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">mutate</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">z_group =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">ifelse</span>(z <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;=</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"z &gt;= 0"</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"z &lt; 0"</span>)) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb30-8">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">ggplot</span>(<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">aes</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">x =</span> x1, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">y =</span> z, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">color =</span> z_group)) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb30-9">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">geom_line</span>() <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb30-10">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">labs</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">title =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Plot of z against x1"</span>,</span>
<span id="cb30-11">       <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">x =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"x1"</span>,</span>
<span id="cb30-12">       <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">y =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"z"</span>,</span>
<span id="cb30-13">       <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">color =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"z group"</span>) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb30-14">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">theme_minimal</span>()</span></code></pre></div></div>
</div>
<div id="tabset-6-10" class="tab-pane" aria-labelledby="tabset-6-10-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb31" style="background: #f1f3f5;"><pre class="sourceCode py code-with-copy"><code class="sourceCode python"><span id="cb31-1"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">import</span> pandas <span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">as</span> pd</span>
<span id="cb31-2"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">import</span> matplotlib.pyplot <span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">as</span> plt</span>
<span id="cb31-3"></span>
<span id="cb31-4"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Assume df already exists with columns: x1, y1, y2, y3</span></span>
<span id="cb31-5"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># -------------------------------------------------</span></span>
<span id="cb31-6"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># 1. Keep only rows where y1 &lt; 2</span></span>
<span id="cb31-7">df <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> df[df[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'y1'</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>].copy()</span>
<span id="cb31-8"></span>
<span id="cb31-9"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># 2. Create a new column z that is the sum of y1, y2, and y3</span></span>
<span id="cb31-10">df[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z'</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> df[[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'y1'</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'y2'</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'y3'</span>]].<span class="bu" style="color: null;
background-color: null;
font-style: inherit;">sum</span>(axis<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>)</span>
<span id="cb31-11"></span>
<span id="cb31-12"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># 3. Plot z vs. x1 with two separate lines:</span></span>
<span id="cb31-13"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">#    - one line for points where z &gt;= 0</span></span>
<span id="cb31-14"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">#    - another line for points where z &lt; 0</span></span>
<span id="cb31-15">plt.figure(figsize<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span>(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">8</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">5</span>))</span>
<span id="cb31-16"></span>
<span id="cb31-17"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Positive‑z line</span></span>
<span id="cb31-18">pos <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> df[df[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z'</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;=</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>]</span>
<span id="cb31-19">plt.plot(pos[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'x1'</span>], pos[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z'</span>],</span>
<span id="cb31-20">         marker<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'o'</span>, linestyle<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'-'</span>, color<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'steelblue'</span>,</span>
<span id="cb31-21">         label<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z ≥ 0'</span>)</span>
<span id="cb31-22"></span>
<span id="cb31-23"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Negative‑z line</span></span>
<span id="cb31-24">neg <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> df[df[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z'</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>]</span>
<span id="cb31-25">plt.plot(neg[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'x1'</span>], neg[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z'</span>],</span>
<span id="cb31-26">         marker<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'o'</span>, linestyle<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'-'</span>, color<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'tomato'</span>,</span>
<span id="cb31-27">         label<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z &lt; 0'</span>)</span>
<span id="cb31-28"></span>
<span id="cb31-29"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Add axis labels, title and legend</span></span>
<span id="cb31-30">plt.xlabel(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'x1'</span>)</span>
<span id="cb31-31">plt.ylabel(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z (y1 + y2 + y3)'</span>)</span>
<span id="cb31-32">plt.title(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z vs. x1 separated by sign of z'</span>)</span>
<span id="cb31-33">plt.legend()</span>
<span id="cb31-34">plt.grid(<span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">True</span>)</span>
<span id="cb31-35">plt.tight_layout()</span>
<span id="cb31-36">plt.show()</span></code></pre></div></div>
</div>
<div id="tabset-6-11" class="tab-pane" aria-labelledby="tabset-6-11-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb32" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb32-1"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">library</span>(tidyverse)</span>
<span id="cb32-2"></span>
<span id="cb32-3">df <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb32-4">  <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Keep rows where y1 is less than 2</span></span>
<span id="cb32-5">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">filter</span>(y1 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb32-6">  <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Add a new column z = y1 + y2 + y3</span></span>
<span id="cb32-7">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">mutate</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">z =</span> y1 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> y2 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> y3) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb32-8">  <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Plot z vs x1 with separate lines for z &gt;= 0 and z &lt; 0</span></span>
<span id="cb32-9">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">ggplot</span>(<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">aes</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">x =</span> x1, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">y =</span> z, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">colour =</span> z <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;=</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>)) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb32-10">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">geom_line</span>() <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb32-11">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">scale_colour_manual</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">name =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"z sign"</span>,</span>
<span id="cb32-12">                      <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">values =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">c</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"TRUE"</span>  <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"blue"</span>,</span>
<span id="cb32-13">                                 <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"FALSE"</span> <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"red"</span>),</span>
<span id="cb32-14">                      <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">labels =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">c</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"z ≥ 0"</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"z &lt; 0"</span>)) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb32-15">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">labs</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">x =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"x1"</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">y =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"z (y1 + y2 + y3)"</span>,</span>
<span id="cb32-16">       <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">title =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"z vs x1 with positive/negative lines"</span>) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb32-17">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">theme_minimal</span>()</span></code></pre></div></div>
</div>
<div id="tabset-6-12" class="tab-pane" aria-labelledby="tabset-6-12-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb33" style="background: #f1f3f5;"><pre class="sourceCode py code-with-copy"><code class="sourceCode python"><span id="cb33-1"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">import</span> pandas <span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">as</span> pd</span>
<span id="cb33-2"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">import</span> matplotlib.pyplot <span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">as</span> plt</span>
<span id="cb33-3"></span>
<span id="cb33-4"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Filter the dataframe where y1 &lt; 2</span></span>
<span id="cb33-5">df_filtered <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> df[df[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'y1'</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>].copy()</span>
<span id="cb33-6"></span>
<span id="cb33-7"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Create a new column 'z' as the sum of y1, y2, y3</span></span>
<span id="cb33-8">df_filtered[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z'</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> df_filtered[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'y1'</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> df_filtered[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'y2'</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> df_filtered[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'y3'</span>]</span>
<span id="cb33-9"></span>
<span id="cb33-10"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Separate data for plotting based on z &gt;= 0 and z &lt; 0</span></span>
<span id="cb33-11">df_positive <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> df_filtered[df_filtered[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z'</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;=</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>]</span>
<span id="cb33-12">df_negative <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> df_filtered[df_filtered[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z'</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>]</span>
<span id="cb33-13"></span>
<span id="cb33-14"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Plot z against x1 with two lines</span></span>
<span id="cb33-15">plt.figure(figsize<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span>(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">10</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">6</span>))</span>
<span id="cb33-16">plt.plot(df_positive[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'x1'</span>], df_positive[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z'</span>], label<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z &gt;= 0'</span>, marker<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'o'</span>)</span>
<span id="cb33-17">plt.plot(df_negative[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'x1'</span>], df_negative[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z'</span>], label<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z &lt; 0'</span>, marker<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'x'</span>)</span>
<span id="cb33-18"></span>
<span id="cb33-19">plt.xlabel(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'x1'</span>)</span>
<span id="cb33-20">plt.ylabel(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'z'</span>)</span>
<span id="cb33-21">plt.title(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'Plot of z vs x1 (filtered by y1 &lt; 2)'</span>)</span>
<span id="cb33-22">plt.legend()</span>
<span id="cb33-23">plt.grid(<span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">True</span>)</span>
<span id="cb33-24">plt.show()</span></code></pre></div></div>
</div>
<div id="tabset-6-13" class="tab-pane" aria-labelledby="tabset-6-13-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb34" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb34-1"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">library</span>(tidyverse)</span>
<span id="cb34-2"></span>
<span id="cb34-3">df_processed <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> df <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb34-4">  <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Keep only rows where y1 is less than 2</span></span>
<span id="cb34-5">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">filter</span>(y1 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb34-6">  <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Create a new column 'z' as the sum of y1, y2, and y3</span></span>
<span id="cb34-7">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">mutate</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">z =</span> y1 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> y2 <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> y3) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb34-8">  <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Create a categorical column to distinguish lines based on z</span></span>
<span id="cb34-9">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">mutate</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">group =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">case_when</span>(</span>
<span id="cb34-10">    z <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;=</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span> <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">~</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Positive or Zero"</span>,</span>
<span id="cb34-11">    z <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span> <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">~</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Negative"</span></span>
<span id="cb34-12">  )) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb34-13">  <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Plot z against x1, coloring by the group condition</span></span>
<span id="cb34-14">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">ggplot</span>(<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">aes</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">x =</span> x1, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">y =</span> z, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">color =</span> group)) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb34-15">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">geom_line</span>() <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb34-16">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">labs</span>(</span>
<span id="cb34-17">    <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">title =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Plot of Z vs X1 (Colored by Sign of Z)"</span>,</span>
<span id="cb34-18">    <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">x =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"x1"</span>,</span>
<span id="cb34-19">    <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">y =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"z"</span>,</span>
<span id="cb34-20">    <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">color =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Z Group"</span></span>
<span id="cb34-21">  ) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb34-22">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">theme_minimal</span>()</span></code></pre></div></div>
</div>
</div>
</div>
<p>What you should see is that it is harder to parse the pandas code, and that even LLM-generated tidyverse codes are often overly complicated and with pointless comments<sup>14</sup>.</p>



<div class="no-row-height column-margin column-container"><div id="fn14"><p><sup>14</sup>&nbsp;Llama 4 Scout actually did well for the tidyverse code, though all of them still use the old <code>%&gt;%</code> syntax for pipes.</p></div></div></section>


<a onclick="window.scrollTo(0, 0); return false;" id="quarto-back-to-top"><i class="bi bi-arrow-up"></i> Back to top</a><div id="quarto-appendix" class="default"><section class="quarto-appendix-contents" id="quarto-bibliography"><h2 class="anchored quarto-appendix-heading">References</h2><div id="refs" class="references csl-bib-body hanging-indent" data-entry-spacing="0">
<div id="ref-WickhamTidy2014" class="csl-entry">
Wickham, Hadley. 2014. <span>“Tidy Data.”</span> <em>Journal of Statistical Software</em> 59 (10). <a href="https://doi.org/10.18637/jss.v059.i10">https://doi.org/10.18637/jss.v059.i10</a>.
</div>
</div></section></div> ]]></description>
  <category>software</category>
  <category>programming</category>
  <guid>https://jianweicheong.codeberg.page/posts/r-heart.html</guid>
  <pubDate>Fri, 12 Jun 2026 16:00:00 GMT</pubDate>
</item>
<item>
  <title>Plotting with live preview in Typst with Lilaq</title>
  <link>https://jianweicheong.codeberg.page/posts/typst-lilaq.html</link>
  <description><![CDATA[ 




<section id="plotting-is-a-chore" class="level1 page-columns page-full">
<h1>Plotting is a chore</h1>
<p>Beyond using the program’s default plotting library, such as in <a href="https://en.wikipedia.org/wiki/MATLAB">MATLAB</a> or <a href="https://en.wikipedia.org/wiki/Wolfram_Mathematica">Mathematica</a>, the most dominant plotting libraries in science are most likely Python’s <a href="https://matplotlib.org/">Matplotlib</a> and R’s <a href="https://ggplot2.tidyverse.org/">ggplot</a>, with Julia’s <a href="https://docs.makie.org/stable/">Makie.jl</a> or <a href="https://docs.juliaplots.org/latest/">Plots.jl</a> steadily gaining traction<sup>1</sup>.</p>
<div class="no-row-height column-margin column-container"><div id="fn1"><p><sup>1</sup>&nbsp;There are of course a myriad of other popular libraries in use, such as <a href="https://seaborn.pydata.org/">Seaborn</a>, <a href="https://plotly.com/python/">Plotly</a>, <a href="https://altair-viz.github.io/">Altair</a>, <a href="https://vega.github.io/vega/">Vega</a>/<a href="https://vega.github.io/vega-lite/">Vega-lite</a>, <a href="https://d3js.org/">D3.js</a>, or even <a href="https://tikz.dev/pgfplots/">PGFplots</a> etc.</p></div></div><p>These are great of course, but plotting with them has always been a chore to me. I don’t know about you, but for me, some of the most boring parts of my work is writing the same templates of codes over and over again. You know, initializing a bunch of empty lists/arrays, looping over some parameters, then adding computation results to them etc.</p>
<p>In the case of plotting, this comes in the form of creating <code>Figure</code> and <code>Axis</code> objects, looping over subplot indices, messing with the levels of the colormap etc. Of course, one can easily plot something quick and dirty, but if I want something complicated or at least presentable, I have to write a bunch of boilerplate code, especially if I want more advanced layouts.</p>
<p>While the elegant <a href="https://en.wikipedia.org/wiki/Wilkinson%27s_Grammar_of_Graphics">grammar of graphics</a> workflow of ggplot2 (and more recently <a href="https://tidierorg.github.io/Tidier.jl/v1.6.1/">Tidier.jl</a>), can ease this pain for complicated plots, some old-schoolers still swear by the simple but powerful <a href="http://gnuplot.info/">gnuplot</a>. Being a lightweight command line program, gnuplot<sup>2</sup> is favored for quick and boilerplate-free plotting.</p>
<div class="no-row-height column-margin column-container"><div id="fn2"><p><sup>2</sup>&nbsp;Pronounced “<em>NEW</em>plot” as in the <a href="https://en.wikipedia.org/wiki/Wildebeest">animal gnu</a> (US pronunciation), and has nothing to do with the <a href="https://en.wikipedia.org/wiki/GNU">GNU project</a>, pronounced “<em>GNOO</em>”.</p></div></div><p>Here’s an example where a nice figure can be plotted with little to no boilerplate code in gnuplot. There’s no importing of libraries or creating of <code>Figure</code> or <code>Axis</code> objects etc.</p>
<div class="column-page-right">
<div class="columns">
<div class="column">
<div class="tabset-margin-container"></div><div class="panel-tabset">
<ul class="nav nav-tabs"><li class="nav-item"><a class="nav-link active" id="tabset-1-1-tab" data-bs-toggle="tab" data-bs-target="#tabset-1-1" aria-controls="tabset-1-1" aria-selected="true" href="">gnuplot</a></li><li class="nav-item"><a class="nav-link" id="tabset-1-2-tab" data-bs-toggle="tab" data-bs-target="#tabset-1-2" aria-controls="tabset-1-2" aria-selected="false" href="">Matplotlib</a></li><li class="nav-item"><a class="nav-link" id="tabset-1-3-tab" data-bs-toggle="tab" data-bs-target="#tabset-1-3" aria-controls="tabset-1-3" aria-selected="false" href="">Makie.jl</a></li><li class="nav-item"><a class="nav-link" id="tabset-1-4-tab" data-bs-toggle="tab" data-bs-target="#tabset-1-4" aria-controls="tabset-1-4" aria-selected="false" href="">ggplot</a></li></ul>
<div class="tab-content">
<div id="tabset-1-1" class="tab-pane active" aria-labelledby="tabset-1-1-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb1" style="background: #f1f3f5;"><pre class="sourceCode sh code-with-copy"><code class="sourceCode bash"><span id="cb1-1"><span class="bu" style="color: null;
background-color: null;
font-style: inherit;">set</span> title <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"This is an example plot"</span></span>
<span id="cb1-2"><span class="bu" style="color: null;
background-color: null;
font-style: inherit;">set</span> xlabel <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"x"</span></span>
<span id="cb1-3"><span class="bu" style="color: null;
background-color: null;
font-style: inherit;">set</span> ylabel <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"y"</span></span>
<span id="cb1-4"><span class="bu" style="color: null;
background-color: null;
font-style: inherit;">set</span> samples 100</span>
<span id="cb1-5"></span>
<span id="cb1-6"><span class="ex" style="color: null;
background-color: null;
font-style: inherit;">plot</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">[-</span><span class="ss" style="color: #20794D;
background-color: null;
font-style: inherit;">10:10</span><span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">]</span> 2<span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">*</span>sin<span class="er" style="color: #AD0000;
background-color: null;
font-style: inherit;">(</span><span class="ex" style="color: null;
background-color: null;
font-style: inherit;">x</span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">)</span><span class="ex" style="color: null;
background-color: null;
font-style: inherit;">/cos</span><span class="er" style="color: #AD0000;
background-color: null;
font-style: inherit;">(</span><span class="ex" style="color: null;
background-color: null;
font-style: inherit;">2*x</span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">)</span> <span class="ex" style="color: null;
background-color: null;
font-style: inherit;">title</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"2sin(x)/cos(2x)"</span></span></code></pre></div></div>
</div>
<div id="tabset-1-2" class="tab-pane" aria-labelledby="tabset-1-2-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb2" style="background: #f1f3f5;"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb2-1"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">import</span> numpy <span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">as</span> np</span>
<span id="cb2-2"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">import</span> matplotlib.pyplot <span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">as</span> plt</span>
<span id="cb2-3"></span>
<span id="cb2-4">x <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> np.linspace(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">10</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">10</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">100</span>)</span>
<span id="cb2-5">y <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> np.sin(x) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">/</span> np.cos(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> x)</span>
<span id="cb2-6"></span>
<span id="cb2-7">plt.figure()</span>
<span id="cb2-8">plt.plot(x, y, label<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="vs" style="color: #20794D;
background-color: null;
font-style: inherit;">r'</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">$</span><span class="vs" style="color: #20794D;
background-color: null;
font-style: inherit;">2</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">\s</span><span class="vs" style="color: #20794D;
background-color: null;
font-style: inherit;">in</span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">(</span><span class="vs" style="color: #20794D;
background-color: null;
font-style: inherit;">x</span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">)</span><span class="vs" style="color: #20794D;
background-color: null;
font-style: inherit;">/</span><span class="er" style="color: #AD0000;
background-color: null;
font-style: inherit;">\</span><span class="vs" style="color: #20794D;
background-color: null;
font-style: inherit;">cos</span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">(</span><span class="vs" style="color: #20794D;
background-color: null;
font-style: inherit;">2x</span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">)</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">$</span><span class="vs" style="color: #20794D;
background-color: null;
font-style: inherit;">'</span>)</span>
<span id="cb2-9">plt.title(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"This is an example plot"</span>)</span>
<span id="cb2-10">plt.xlabel(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"x"</span>)</span>
<span id="cb2-11">plt.ylabel(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"y"</span>)</span>
<span id="cb2-12">plt.legend()</span></code></pre></div></div>
</div>
<div id="tabset-1-3" class="tab-pane" aria-labelledby="tabset-1-3-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb3" style="background: #f1f3f5;"><pre class="sourceCode julia code-with-copy"><code class="sourceCode julia"><span id="cb3-1"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">using</span> <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">CairoMakie</span></span>
<span id="cb3-2"></span>
<span id="cb3-3">x <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">range</span>(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">10</span>, <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">10</span>, <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">100</span>)</span>
<span id="cb3-4">y <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> @. <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">sin</span>(x) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">/</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">cos</span>(<span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>x)</span>
<span id="cb3-5"></span>
<span id="cb3-6">fig <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">Figure</span>()</span>
<span id="cb3-7">ax <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">Axis</span>(fig[<span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>, <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>],</span>
<span id="cb3-8">    title <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"This is an example plot"</span>,</span>
<span id="cb3-9">    xlabel <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"x"</span>,</span>
<span id="cb3-10">    ylabel <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"y"</span>)</span>
<span id="cb3-11"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">lines!</span>(ax, x, y, label <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> L<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"2\sin(x)/\cos(2x)"</span>)</span>
<span id="cb3-12"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">axislegend</span>()</span></code></pre></div></div>
</div>
<div id="tabset-1-4" class="tab-pane" aria-labelledby="tabset-1-4-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb4" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb4-1"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">library</span>(ggplot2)</span>
<span id="cb4-2"></span>
<span id="cb4-3">x <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">seq</span>(<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">10</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">10</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">length.out =</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">100</span>)</span>
<span id="cb4-4">y <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span> <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">sin</span>(x) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">/</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">cos</span>(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span> <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> x)</span>
<span id="cb4-5">df <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">data.frame</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">x =</span> x, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">y =</span> y)</span>
<span id="cb4-6"></span>
<span id="cb4-7"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">ggplot</span>(df, <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">aes</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">x =</span> x, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">y =</span> y)) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb4-8">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">geom_line</span>() <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb4-9">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">labs</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">title =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"This is an example plot"</span>,</span>
<span id="cb4-10">       <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">x =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"x"</span>,</span>
<span id="cb4-11">       <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">y =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"y"</span>,</span>
<span id="cb4-12">       <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">caption =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">expression</span>(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span> <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">sin</span>(x) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">/</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">cos</span>(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span> <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> x)))</span></code></pre></div></div>
</div>
</div>
</div>
</div><div class="column">
<p><img src="https://jianweicheong.codeberg.page/media/posts/typst-lilaq/gnuplot.svg" class="img-fluid"></p>
</div>
</div>
</div>
<p>And a slightly more complicated example where we plot data from a CSV file (I don’t even want to write the Matplotlib, Makie.jl, and ggplot versions of this).</p>
<div class="column-page-right">
<div class="columns">
<div class="column">
<div class="tabset-margin-container"></div><div class="panel-tabset">
<ul class="nav nav-tabs"><li class="nav-item"><a class="nav-link active" id="tabset-2-1-tab" data-bs-toggle="tab" data-bs-target="#tabset-2-1" aria-controls="tabset-2-1" aria-selected="true" href="">gnuplot</a></li></ul>
<div class="tab-content">
<div id="tabset-2-1" class="tab-pane active" aria-labelledby="tabset-2-1-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb5" style="background: #f1f3f5;"><pre class="sourceCode sh code-with-copy"><code class="sourceCode bash"><span id="cb5-1"><span class="bu" style="color: null;
background-color: null;
font-style: inherit;">set</span> nokey  <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># turn off legend</span></span>
<span id="cb5-2"><span class="bu" style="color: null;
background-color: null;
font-style: inherit;">set</span> datafile separator <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">","</span></span>
<span id="cb5-3"></span>
<span id="cb5-4"><span class="bu" style="color: null;
background-color: null;
font-style: inherit;">set</span> pm3d at bs  <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># plot at bottom and surface</span></span>
<span id="cb5-5"><span class="bu" style="color: null;
background-color: null;
font-style: inherit;">set</span> dgrid3d 100,100</span>
<span id="cb5-6"></span>
<span id="cb5-7"><span class="bu" style="color: null;
background-color: null;
font-style: inherit;">set</span> multiplot layout 1,2</span>
<span id="cb5-8"><span class="ex" style="color: null;
background-color: null;
font-style: inherit;">splot</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"data.csv"</span> using 2:3:5 with pm3d</span>
<span id="cb5-9"><span class="ex" style="color: null;
background-color: null;
font-style: inherit;">splot</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"data.csv"</span> using 2:3:12 with pm3d</span>
<span id="cb5-10"><span class="bu" style="color: null;
background-color: null;
font-style: inherit;">unset</span> <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">multiplot</span></span></code></pre></div></div>
</div>
</div>
</div>
</div><div class="column">
<p><img src="https://jianweicheong.codeberg.page/media/posts/typst-lilaq/gnuplot2.svg" class="img-fluid"></p>
</div>
</div>
</div>
<p>Being a separate program from where your data are typically computed/generated<sup>3</sup>, gnuplot requires one to save their data out into some data format, such as CSV, before loading into gnuplot to perform the plotting. Putting aside the fact that this is good programming practice for scientists in general, this is powerful because the same gnuplot script can quickly visualize different datasets in a consistent manner.</p>
<div class="no-row-height column-margin column-container"><div id="fn3"><p><sup>3</sup>&nbsp;Though there’s actually various libraries that calls gnuplot without leaving your favorite programming language, e.g., <a href="https://github.com/dkogan/gnuplotlib">gnuplotlib</a>, <a href="https://mbaz.github.io/Gaston.jl/v2/">Gaston.jl</a>, <a href="https://github.com/SiegeLord/RustGnuplot">RustGnuplot</a>, and <a href="https://en.wikipedia.org/wiki/Gnuplot#Programming_and_application_interfaces">various others</a>.</p></div><div id="fn4"><p><sup>4</sup>&nbsp;Interestingly, there is now also <a href="https://gridpaper.org/">Gridpaper</a> by <a href="https://harishnarayanan.org/">Harish Narayanan</a>, which uses gnuplot compiled to <a href="https://en.wikipedia.org/wiki/WebAssembly">WebAssembly</a> under the hood.</p></div></div><p>Nevertheless, even though I love gnuplot, it shares a common problem with the other plotting libraries (and most programming languages) in that iterating between plots are slow. Maybe I am spoilt by the <a href="https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop"><em>Read-eval-print loop</em></a> of Lisp, but I wish plotting is more interactive or responsive, like <a href="https://www.desmos.com/calculator">Desmos</a><sup>4</sup>, instead of the same old: save file, run it, wait, and repeat because you don’t like how it looks. Of course, one can set up <a href="https://ipywidgets.readthedocs.io/en/latest/">Jupyter notebook widgets</a> or a <a href="https://shiny.posit.co/">Shiny dashboard</a> for interactivity, but then we are back to writing boilerplate codes.</p>
</section>
<section id="enters-typst-and-lilaq" class="level1 page-columns page-full">
<h1>Enters Typst and Lilaq</h1>
<p>I have mentioned <a href="https://github.com/typst/typst">Typst</a> before, as well as the plotting library <a href="https://lilaq.org/">Lilaq</a>, lauding the former for its fast live preview due to its incremental compilation. Specifically, the <a href="https://myriad-dreamin.github.io/tinymist/">tinymist</a> <a href="https://en.wikipedia.org/wiki/Language_Server_Protocol">LSP</a> is able to trigger the preview “<a href="https://myriad-dreamin.github.io/tinymist/feature/preview.html">on type</a>”, i.e., with every keystroke<sup>5</sup>.</p>
<div class="no-row-height column-margin column-container"><div id="fn5"><p><sup>5</sup>&nbsp;Or one can use the <a href="https://typst.app/">web app</a> which also has instant live preview.</p></div></div><div class="quarto-figure quarto-figure-center">
<figure class="figure">
<p><video src="../media/posts/typst-lilaq/webapp.webm" class="img-fluid" controls=""><a href="../media/posts/typst-lilaq/webapp.webm">Video</a></video></p>
<figcaption>Plotting with instant live preview in the web app.</figcaption>
</figure>
</div>
<div class="column-page-right">
<div class="columns">
<div class="column">
<div class="tabset-margin-container"></div><div class="panel-tabset">
<ul class="nav nav-tabs"><li class="nav-item"><a class="nav-link active" id="tabset-3-1-tab" data-bs-toggle="tab" data-bs-target="#tabset-3-1" aria-controls="tabset-3-1" aria-selected="true" href="">Lilaq</a></li><li class="nav-item"><a class="nav-link" id="tabset-3-2-tab" data-bs-toggle="tab" data-bs-target="#tabset-3-2" aria-controls="tabset-3-2" aria-selected="false" href="">gnuplot</a></li><li class="nav-item"><a class="nav-link" id="tabset-3-3-tab" data-bs-toggle="tab" data-bs-target="#tabset-3-3" aria-controls="tabset-3-3" aria-selected="false" href="">Matplotlib</a></li><li class="nav-item"><a class="nav-link" id="tabset-3-4-tab" data-bs-toggle="tab" data-bs-target="#tabset-3-4" aria-controls="tabset-3-4" aria-selected="false" href="">Makie.jl</a></li><li class="nav-item"><a class="nav-link" id="tabset-3-5-tab" data-bs-toggle="tab" data-bs-target="#tabset-3-5" aria-controls="tabset-3-5" aria-selected="false" href="">ggplot</a></li></ul>
<div class="tab-content">
<div id="tabset-3-1" class="tab-pane active" aria-labelledby="tabset-3-1-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb6" style="background: #f1f3f5;"><pre class="sourceCode typst code-with-copy"><code class="sourceCode typst"><span id="cb6-1"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">#</span><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">import</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"@preview/lilaq:0.5.0"</span> as lq</span>
<span id="cb6-2"></span>
<span id="cb6-3"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">#set</span> <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">page</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>margin<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0.5em</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> height<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">auto</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> width<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">auto</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span id="cb6-4"></span>
<span id="cb6-5"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">#let</span> <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">xs</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> lq<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>linspace<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(-</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">10</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">10</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> num<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">100</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span id="cb6-6"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">#let</span> <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">ys</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> xs<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>map<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>x <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=&gt;</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>*calc<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>sin<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>x<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span> / calc<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>cos<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>*x<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">))</span></span>
<span id="cb6-7"></span>
<span id="cb6-8"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">#</span>lq<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>diagram<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span></span>
<span id="cb6-9">  lq<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>plot<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>xs<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> ys<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> label<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="ss" style="color: #20794D;
background-color: null;
font-style: inherit;">$2sin(x)\/cos(2x)$</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">),</span></span>
<span id="cb6-10">  xlabel<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="ss" style="color: #20794D;
background-color: null;
font-style: inherit;">$x$</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb6-11">  ylabel<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="ss" style="color: #20794D;
background-color: null;
font-style: inherit;">$y$</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span></span>
<span id="cb6-12">  title<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"This is an example plot"</span></span>
<span id="cb6-13"><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span></code></pre></div></div>
</div>
<div id="tabset-3-2" class="tab-pane" aria-labelledby="tabset-3-2-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb7" style="background: #f1f3f5;"><pre class="sourceCode sh code-with-copy"><code class="sourceCode bash"><span id="cb7-1"><span class="bu" style="color: null;
background-color: null;
font-style: inherit;">set</span> title <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"This is an example plot"</span></span>
<span id="cb7-2"><span class="bu" style="color: null;
background-color: null;
font-style: inherit;">set</span> xlabel <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"x"</span></span>
<span id="cb7-3"><span class="bu" style="color: null;
background-color: null;
font-style: inherit;">set</span> ylabel <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"y"</span></span>
<span id="cb7-4"><span class="bu" style="color: null;
background-color: null;
font-style: inherit;">set</span> samples 100</span>
<span id="cb7-5"></span>
<span id="cb7-6"><span class="ex" style="color: null;
background-color: null;
font-style: inherit;">plot</span> <span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">[-</span><span class="ss" style="color: #20794D;
background-color: null;
font-style: inherit;">10:10</span><span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">]</span> 2<span class="pp" style="color: #AD0000;
background-color: null;
font-style: inherit;">*</span>sin<span class="er" style="color: #AD0000;
background-color: null;
font-style: inherit;">(</span><span class="ex" style="color: null;
background-color: null;
font-style: inherit;">x</span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">)</span><span class="ex" style="color: null;
background-color: null;
font-style: inherit;">/cos</span><span class="er" style="color: #AD0000;
background-color: null;
font-style: inherit;">(</span><span class="ex" style="color: null;
background-color: null;
font-style: inherit;">2*x</span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">)</span> <span class="ex" style="color: null;
background-color: null;
font-style: inherit;">title</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"2sin(x)/cos(2x)"</span></span></code></pre></div></div>
</div>
<div id="tabset-3-3" class="tab-pane" aria-labelledby="tabset-3-3-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb8" style="background: #f1f3f5;"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb8-1"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">import</span> numpy <span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">as</span> np</span>
<span id="cb8-2"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">import</span> matplotlib.pyplot <span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">as</span> plt</span>
<span id="cb8-3"></span>
<span id="cb8-4">x <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> np.linspace(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">10</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">10</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">100</span>)</span>
<span id="cb8-5">y <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> np.sin(x) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">/</span> np.cos(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> x)</span>
<span id="cb8-6"></span>
<span id="cb8-7">plt.figure()</span>
<span id="cb8-8">plt.plot(x, y, label<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="vs" style="color: #20794D;
background-color: null;
font-style: inherit;">r'</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">$</span><span class="vs" style="color: #20794D;
background-color: null;
font-style: inherit;">2</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">\s</span><span class="vs" style="color: #20794D;
background-color: null;
font-style: inherit;">in</span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">(</span><span class="vs" style="color: #20794D;
background-color: null;
font-style: inherit;">x</span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">)</span><span class="vs" style="color: #20794D;
background-color: null;
font-style: inherit;">/</span><span class="er" style="color: #AD0000;
background-color: null;
font-style: inherit;">\</span><span class="vs" style="color: #20794D;
background-color: null;
font-style: inherit;">cos</span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">(</span><span class="vs" style="color: #20794D;
background-color: null;
font-style: inherit;">2x</span><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">)</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">$</span><span class="vs" style="color: #20794D;
background-color: null;
font-style: inherit;">'</span>)</span>
<span id="cb8-9">plt.title(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"This is an example plot"</span>)</span>
<span id="cb8-10">plt.xlabel(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"x"</span>)</span>
<span id="cb8-11">plt.ylabel(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"y"</span>)</span>
<span id="cb8-12">plt.legend()</span></code></pre></div></div>
</div>
<div id="tabset-3-4" class="tab-pane" aria-labelledby="tabset-3-4-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb9" style="background: #f1f3f5;"><pre class="sourceCode julia code-with-copy"><code class="sourceCode julia"><span id="cb9-1"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">using</span> <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">CairoMakie</span></span>
<span id="cb9-2"></span>
<span id="cb9-3">x <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">range</span>(<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">10</span>, <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">10</span>, <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">100</span>)</span>
<span id="cb9-4">y <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> @. <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">sin</span>(x) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">/</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">cos</span>(<span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>x)</span>
<span id="cb9-5"></span>
<span id="cb9-6">fig <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">Figure</span>()</span>
<span id="cb9-7">ax <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">Axis</span>(fig[<span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>, <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>],</span>
<span id="cb9-8">    title <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"This is an example plot"</span>,</span>
<span id="cb9-9">    xlabel <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"x"</span>,</span>
<span id="cb9-10">    ylabel <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"y"</span>)</span>
<span id="cb9-11"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">lines!</span>(ax, x, y, label <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> L<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"2\sin(x)/\cos(2x)"</span>)</span>
<span id="cb9-12"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">axislegend</span>()</span></code></pre></div></div>
</div>
<div id="tabset-3-5" class="tab-pane" aria-labelledby="tabset-3-5-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb10" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb10-1"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">library</span>(ggplot2)</span>
<span id="cb10-2"></span>
<span id="cb10-3">x <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">seq</span>(<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">10</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">10</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">length.out =</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">100</span>)</span>
<span id="cb10-4">y <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span> <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">sin</span>(x) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">/</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">cos</span>(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span> <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> x)</span>
<span id="cb10-5">df <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">data.frame</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">x =</span> x, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">y =</span> y)</span>
<span id="cb10-6"></span>
<span id="cb10-7"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">ggplot</span>(df, <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">aes</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">x =</span> x, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">y =</span> y)) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb10-8">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">geom_line</span>() <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb10-9">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">labs</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">title =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"This is an example plot"</span>,</span>
<span id="cb10-10">       <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">x =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"x"</span>,</span>
<span id="cb10-11">       <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">y =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"y"</span>,</span>
<span id="cb10-12">       <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">caption =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">expression</span>(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span> <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">sin</span>(x) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">/</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">cos</span>(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span> <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> x)))</span></code></pre></div></div>
</div>
</div>
</div>
</div><div class="column">
<p><img src="https://jianweicheong.codeberg.page/media/posts/typst-lilaq/lilaq.svg" class="img-fluid"></p>
</div>
</div>
</div>
<p>I would argue that while we sacrificed some conciseness and brevity for responsiveness as compared to gnuplot, this is a little better than Matplotlib and Makie.jl. Not to mention getting access to first-class Math support like LaTeX as we are using Typst.</p>
<p>As for plotting from a CSV file,</p>
<div class="column-page-right">
<div class="columns">
<div class="column">
<div class="tabset-margin-container"></div><div class="panel-tabset">
<ul class="nav nav-tabs"><li class="nav-item"><a class="nav-link active" id="tabset-4-1-tab" data-bs-toggle="tab" data-bs-target="#tabset-4-1" aria-controls="tabset-4-1" aria-selected="true" href="">Lilaq</a></li><li class="nav-item"><a class="nav-link" id="tabset-4-2-tab" data-bs-toggle="tab" data-bs-target="#tabset-4-2" aria-controls="tabset-4-2" aria-selected="false" href="">gnuplot</a></li></ul>
<div class="tab-content">
<div id="tabset-4-1" class="tab-pane active" aria-labelledby="tabset-4-1-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb11" style="background: #f1f3f5;"><pre class="sourceCode typst code-with-copy"><code class="sourceCode typst"><span id="cb11-1"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">#</span><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">import</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"@preview/lilaq:0.5.0"</span> as lq</span>
<span id="cb11-2"></span>
<span id="cb11-3"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">#set</span> <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">page</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>margin<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0.5em</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> height<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">auto</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> width<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">auto</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span id="cb11-4"></span>
<span id="cb11-5"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">#let</span> <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">data</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> lq<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>load-txt<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>read<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"data.csv"</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">),</span> header<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">true</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span id="cb11-6"></span>
<span id="cb11-7"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">#let</span> <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">z1</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> array<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>zip<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>..data<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>E<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>chunks<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">100</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> exact<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">true</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">))</span></span>
<span id="cb11-8"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">#let</span> <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">z2</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> array<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>zip<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>..data<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>Em<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>chunks<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">100</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">,</span> exact<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span> <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">true</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">))</span></span>
<span id="cb11-9"></span>
<span id="cb11-10"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">#let</span> <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">colormesh1</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> lq<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>colormesh<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>data<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>xA<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>dedup<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(),</span> data<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>xB<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>dedup<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(),</span> z1<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span id="cb11-11"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">#let</span> <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">colormesh2</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> lq<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>colormesh<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>data<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>xA<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>dedup<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(),</span> data<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>xB<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>dedup<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(),</span> z2<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span id="cb11-12"></span>
<span id="cb11-13"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">#</span>lq<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>diagram<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>colormesh1<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span id="cb11-14"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">#</span>lq<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>colorbar<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>colormesh1<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span id="cb11-15"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">#</span>lq<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>diagram<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>colormesh2<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span>
<span id="cb11-16"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">#</span>lq<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">.</span>colorbar<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">(</span>colormesh2<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">)</span></span></code></pre></div></div>
</div>
<div id="tabset-4-2" class="tab-pane" aria-labelledby="tabset-4-2-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb12" style="background: #f1f3f5;"><pre class="sourceCode sh code-with-copy"><code class="sourceCode bash"><span id="cb12-1"><span class="bu" style="color: null;
background-color: null;
font-style: inherit;">set</span> nokey</span>
<span id="cb12-2"><span class="bu" style="color: null;
background-color: null;
font-style: inherit;">set</span> datafile separator <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">","</span></span>
<span id="cb12-3"></span>
<span id="cb12-4"><span class="bu" style="color: null;
background-color: null;
font-style: inherit;">set</span> pm3d at b</span>
<span id="cb12-5"><span class="bu" style="color: null;
background-color: null;
font-style: inherit;">set</span> view map</span>
<span id="cb12-6"><span class="bu" style="color: null;
background-color: null;
font-style: inherit;">set</span> dgrid3d 100,100</span>
<span id="cb12-7"></span>
<span id="cb12-8"><span class="bu" style="color: null;
background-color: null;
font-style: inherit;">set</span> multiplot layout 1,2</span>
<span id="cb12-9"><span class="ex" style="color: null;
background-color: null;
font-style: inherit;">splot</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"data.csv"</span> using 2:3:5 with pm3d</span>
<span id="cb12-10"><span class="ex" style="color: null;
background-color: null;
font-style: inherit;">splot</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"data.csv"</span> using 2:3:12 with pm3d</span>
<span id="cb12-11"><span class="bu" style="color: null;
background-color: null;
font-style: inherit;">unset</span> <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">multiplot</span></span></code></pre></div></div>
</div>
</div>
</div>
</div><div class="column">
<p><img src="https://jianweicheong.codeberg.page/media/posts/typst-lilaq/lilaq2.svg" class="img-fluid"></p>
</div>
</div>
</div>
<p>Lilaq is still a little verbose, particularly the reshaping of the data into a 2D array for the plotting of the colormesh in this case, which gnuplot takes care of automatically with <code>set dgrid3d 100,100</code>.</p>
<p>Still, the interactivity and responsiveness is great especially for data exploratory work.</p>
<div class="quarto-figure quarto-figure-center">
<figure class="figure">
<p><video src="../media/posts/typst-lilaq/webapp2.webm" class="img-fluid" controls=""><a href="../media/posts/typst-lilaq/webapp2.webm">Video</a></video></p>
<figcaption>Iterating between plots instantly.</figcaption>
</figure>
</div>
</section>
<section id="is-it-really-that-important" class="level1 page-columns page-full">
<h1>Is it really that important?</h1>
<p>To be honest, not really.</p>
<p>In scientific computing, many encounters the “<a href="https://discourse.julialang.org/t/two-language-problem-what-is-it/82925">two-language problem</a>”, where the initial prototyping and exploratory research work is performed in a programming language that is highly productive or interactive, such as Python, R, or MATLAB.</p>
<p>However, when the exploratory work is done and when one require fast number crunching (say in a supercomputer), one has to rewrite the program (or at least the computationally expensive parts) in a high-performance language such as C, C++, Fortran, or Rust.</p>
<p>Julia <a href="https://docs.julialang.org/en/v1/#man-introduction">aims to solve this problem</a> with just-in-time (<a href="https://en.wikipedia.org/wiki/Just-in-time_compilation">JIT</a>) compilation and <a href="https://en.wikipedia.org/wiki/LLVM">LLVM</a><sup>6</sup>, combining the syntax and interactivity of dynamic languages, with the performance and speed of statically-typed languages.</p>
<div class="no-row-height column-margin column-container"><div id="fn6"><p><sup>6</sup>&nbsp;There is now also <a href="https://lfortran.org/">LFortran</a> that attempts the same, also with LLVM.</p></div></div><p>While the context is obviously not the same, you could say that I’m looking for a <strong>“Julia” of plotting</strong>. Using plotting libraries such as Matplotlib for complicated plots is like the worst of both worlds: bloated code yet (relatively) slow iteration of plots.</p>
<p>Why can’t we have the best of both worlds instead: <strong>plot with stupid simple syntax and see the results instantly as we change the code</strong>? I believe gnuplot and Lilaq are the closest we have to this right now, approaching this ideal from the two opposite ends of simple syntax and instant preview, respectively.</p>


</section>


<a onclick="window.scrollTo(0, 0); return false;" id="quarto-back-to-top"><i class="bi bi-arrow-up"></i> Back to top</a> ]]></description>
  <category>programming</category>
  <category>software</category>
  <guid>https://jianweicheong.codeberg.page/posts/typst-lilaq.html</guid>
  <pubDate>Sat, 28 Feb 2026 16:00:00 GMT</pubDate>
</item>
<item>
  <title>Scientific colormaps and why you should care</title>
  <link>https://jianweicheong.codeberg.page/posts/colormaps.html</link>
  <description><![CDATA[ 




<section id="a-case-study" class="level1 page-columns page-full">
<h1>A case study</h1>
<p>What’s the difference between these two colormaps?</p>
<p>Which do we choose to plot our results in?</p>
<div class="quarto-figure quarto-figure-center">
<figure class="figure">
<p><img src="https://jianweicheong.codeberg.page/media/posts/scientific-colormaps/jet_turbo.svg" class="img-fluid figure-img"></p>
<figcaption>Two similar looking colormaps.</figcaption>
</figure>
</div>
<p>Despite the fact that visualizations of scientific results or data are enshrined in journals that lives on after we are gone, most of us put little thoughts on the colormaps that we plot our results in.</p>
<p>I mean, who has the time to care about this? Much less two that hardly look any different! Most will just stick to the default that their plotting programs/libraries chose. In fact, the first colormap of the two was the default used by Matlab and Matplotlib up until around 2015.</p>
<p>But what if I told you that these two colormaps look quite different to people with <a href="https://en.wikipedia.org/wiki/Color_blindness">color vision deficiency (CVD)</a>, and that the first colormap can and has been shown to <strong>misrepresent scientific results</strong>?</p>
<p>The first colormap is the infamous <code>jet</code> (sometimes referred to as “rainbow”), and people have been pleading for the end of its use for a long time<sup>1</sup>. In the extreme, it might even lead to harmful consequences such as medical diagnostic errors <span class="citation" data-cites="BorkinEvaluation2011">(Borkin et al. 2011)</span>!</p>
<div class="no-row-height column-margin column-container"><div id="fn1"><p><sup>1</sup>&nbsp;See <a href="https://www.kennethmoreland.com/color-maps/">this</a>, <a href="https://jakevdp.github.io/blog/2014/10/16/how-bad-is-your-colormap/">this</a>, or <a href="https://www.nature.com/articles/519291d">this</a>, and <a href="https://www.climate-lab-book.ac.uk/2014/end-of-the-rainbow/">this</a>, <a href="https://medvis.org/2012/08/21/rainbow-colormaps-what-are-they-good-for-absolutely-nothing/">this</a>, <a href="https://brushingupscience.com/2015/12/15/stop-using-rainbow-colourmaps/">this</a>, or <a href="https://agilescientific.com/blog/2017/12/14/no-more-rainbows">this</a>. Seriously, there are a lot.</p></div><div id="fn2"><p><sup>2</sup>&nbsp;Non-uniformity is akin to having a non-uniform scale such as the log-scale, e.g., the <code>jet</code> colormap has a long region of blue which rapidly jumps to cyan then green. The two bright bands of cyan and yellow are also examples of non-linearity.</p></div></div><p>As I found out recently, the use of colormaps in scientific visualization is actually quite well-studied <span class="citation" data-cites="LandiniColor2011 KovesiColour2015 ZhouSurvey2016">(Landini and Perryer 2011; Kovesi 2015; Zhou and Hansen 2016)</span>, and it is widely accepted that <strong>colormaps should be perceptually uniform, linear, and CVD-friendly</strong> (or continue to be perceptually uniform and linear for people with CVD)<sup>2</sup>.</p>
<p>The <code>jet</code> colormap fails all three conditions, and thus can easily mislead or misrepresent scientific results. Small differences in values are emphasized if it happens to lie on the cyan-green-yellow region, but greater differences that lies along the blue or red regions are hidden instead. It can also create misleading boundaries in results where there are none due to the sharp cyan and yellow regions.</p>
<p>These shortcomings are why Matlab and Matplotlib eventually developed new colormaps as replacements around 2015; Matlab developed <a href="https://www.mathworks.com/help/matlab/ref/parula.html"><code>parula</code></a>, and Matplotlib made the now popular <a href="https://bids.github.io/colormap/"><code>viridis</code> and a few others</a><sup>3</sup>.</p>
<div class="no-row-height column-margin column-container"><div id="fn3"><p><sup>3</sup>&nbsp;These <a href="https://www.youtube.com/watch?v=xAoljeRJ3lU">two</a> <a href="https://www.youtube.com/watch?v=XjHzLUnHeM0">talks</a> at the SciPy 2015 conference are great if you want to know more about the problems of “rainbow” colormaps, and on what makes a good colormap for scientific visualization.</p></div></div><p>But yet, <code>jet</code> remains somewhat prevalent as <a href="https://brushingupscience.com/2019/10/01/default-colormaps-are-parula-and-viridis-really-an-improvement-over-jet/">some researchers find it hard to give up on it</a> (I spotted some in the wild as recent as this year). This <a href="https://research.google/blog/turbo-an-improved-rainbow-colormap-for-visualization/">led Google to develop the second colormap above, <code>turbo</code></a>, which retains the rainbow color but is supposed to be an improvement over <code>jet</code>. Though it is still not ideal and <a href="https://mpetroff.net/2019/08/discernibility-of-rainbow-colormaps/">there are some criticisms of it</a>. One user on HackerNews even <a href="https://news.ycombinator.com/item?id=20754998">joked that</a>:</p>
<blockquote class="blockquote">
<p>“One good use for this new color map might be: ‘My ignorant boss keeps insisting on using the jet color map even though it is terrible. I can drop this one in and he won’t notice the difference but it will be somewhat better, even if still problematical in many ways.’”</p>
</blockquote>
<p>You see, I believe that it is our responsibility to not only ensure that scientific results are not misrepresented, but also to ensure that it is accessible to people with CVD.</p>
<p>A typical plotting library will provide a myriad of different colormaps, and one should have some restrain to not choose a colormap too freely.</p>
</section>
<section id="so-which-colormap" class="level1 page-columns page-full">
<h1>So which colormap?</h1>
<p>Neither <code>jet</code> nor <code>turbo</code>. Just do not use “rainbow” colormaps (in 99.9% of all cases) regardless of whether you need a sequential or diverging colormap.</p>
<p>But a lot of these programs favors quantity over quality. How do I know if that pretty colormap my favorite plotting library provides is perceptually uniform, linear, and CVD-friendly?</p>
<p>Generally, one just have to know the families of colormaps that were designed with these standards in mind, and plotting libraries with good documentations tend to properly group them into these families, e.g., take a look at <a href="https://juliagraphics.github.io/ColorSchemes.jl/stable/catalogue/">Julia’s</a> where anything grouped under “scientific” or “colorcet” is generally fine.</p>
<p>Nevertheless, here are some general recommendations:</p>
<p><a href="https://matplotlib.org/stable/users/explain/colors/colormaps.html#sequential">Matplotlib’s default</a>, <code>viridis</code>, and a CVD-friendlier version by <span class="citation" data-cites="NunezOptimizing2018">Nuñez, Anderton, and Renslow (2018)</span>, <code>cividis</code>, which I prefer, will suffice for most sequential plots. Though, <code>batlow</code> by <a href="https://www.fabiocrameri.ch/colourmaps/">Fabio Crameri</a> is another favorite and is scientifically derived <span class="citation" data-cites="CrameriThe2020">(Crameri, Shephard, and Heron 2020)</span>. <a href="https://colorcet.com/">ColorCET</a> by <span class="citation" data-cites="KovesiColour2015">Kovesi (2015)</span> also has the CVD-friendly colormap <code>CET-CBL1</code>. All these are good and are widely available in various programs and plotting libraries.</p>
<div class="quarto-figure quarto-figure-center">
<figure class="figure">
<p><img src="https://jianweicheong.codeberg.page/media/posts/scientific-colormaps/colormaps_seq.svg" class="img-fluid figure-img"></p>
<figcaption>Sequential colormaps.</figcaption>
</figure>
</div>

<div class="no-row-height column-margin column-container"><div class="">
<p><a href="https://media.springernature.com/lw685/springer-static/image/art%3A10.1038%2Fs41467-020-19160-7/MediaObjects/41467_2020_19160_Fig1_HTML.png?as=webp" class="lightbox" data-gallery="quarto-lightbox-gallery-1"><img src="https://media.springernature.com/lw685/springer-static/image/art%3A10.1038%2Fs41467-020-19160-7/MediaObjects/41467_2020_19160_Fig1_HTML.png?as=webp" class="img-fluid"></a></p>
<p>Reprinted from <span class="citation" data-cites="CrameriThe2020">Crameri, Shephard, and Heron (2020)</span>.</p>
<p>Comparison between <code>jet</code> and <code>batlow</code>.</p>
<p>Also check out <a href="https://www.youtube.com/watch?v=iDPzWARbFrs">his talk</a> at the University of Oslo <a href="https://www.mn.uio.no/geo/english/about/organisation/geohyd/events/archive/2021/">GEOHYD Lunch Seminar 2021</a>.</p>
</div></div><p>For diverging colormaps, there’s <a href="https://colorbrewer2.org/">ColorBrewer’s</a> <code>RdBu</code>, ColorCET’s <code>CET-D07</code>, or the CVD-friendlier <code>CET-CBD1</code>, though you can’t really go wrong with the rest of the colormaps by Fabio Crameri either (Matplotlib provides Crameri’s <code>berlin</code>, <code>managua</code>, and <code>vanimo</code>).</p>
<div class="quarto-figure quarto-figure-center">
<figure class="figure">
<p><img src="https://jianweicheong.codeberg.page/media/posts/scientific-colormaps/colormaps_div.svg" class="img-fluid figure-img"></p>
<figcaption>Diverging colormaps.</figcaption>
</figure>
</div>
<p>Crameri also has some bi-sequential colormaps that are useful for topographical plots. The only other I know of is <a href="https://matplotlib.org/cmocean/">cmocean’s</a> <code>topo</code>.</p>
<div class="quarto-figure quarto-figure-center">
<figure class="figure">
<p><img src="https://jianweicheong.codeberg.page/media/posts/scientific-colormaps/colormaps_bi.svg" class="img-fluid figure-img"></p>
<figcaption>Bi-sequential colormaps.</figcaption>
</figure>
</div>
<p>Overall, when in doubt, stick to <code>viridis</code>, <code>cividis</code>, and the colormaps by Crameri.</p>
</section>
<section id="what-about-color-cyclessequences" class="level1 page-columns page-full">
<h1>What about color cycles/sequences?</h1>
<p>Color cycles/sequences too are important in scientific visualization given how much more common qualitative/categorical plots are. The only concern is to be perceptually distinguishable, even for people with CVD. However, there are very little consensus on them, and it seems that different programs or libraries often pick their own based on subjective reasoning or aesthetics.</p>
<p>For example, the familiar Matplotlib’s palette (also used by <a href="https://vega.github.io/vega/docs/schemes/#category10">Vega</a>, <a href="https://github.com/d3/d3-3.x-api-reference/blob/master/Ordinal-Scales.md#category10">d3</a>, <a href="https://plotly.com/python/discrete-color/#color-sequences-in-plotly-express">plotly</a> and various others) often seen in published plots were simply <a href="https://matplotlib.org/stable/users/prev_whats_new/dflt_style_changes.html#colors-in-default-property-cycle">adopted from Tableau</a><sup>4</sup>, and <a href="https://dovydas.com/blog/colorblind-friendly-diagrams">is not CVD-friendly</a> despite its widespread usage! A number of the colors in the palette are not perceptually distinct to people with CVD (see “Category 10” in the margin figure below).</p>
<div class="no-row-height column-margin column-container"><div id="fn4"><p><sup>4</sup>&nbsp;Tableau has since <a href="https://www.tableau.com/blog/colors-upgrade-tableau-10-56782">changed to a newer palette</a> based yet again on subjective senses of aesthetic.</p></div></div><p>Still, there have been some attempts to look for CVD-friendly color cycles. For example, the palettes by <a href="https://sronpersonalpages.nl/~pault/#sec:qualitative">Paul Tol</a> and the ones by <a href="https://web.archive.org/web/20030821055411/http://jfly.iam.u-tokyo.ac.jp/color/text.html">Okabe and Ito</a> are quite well-known, especially after the latter was brought to light again by <span class="citation" data-cites="WongPoints2011">Wong (2011)</span> (and thus often misattributed to him, e.g., in Julia’s <a href="https://docs.makie.org/stable/explanations/colors#Colormaps">Makie.jl</a>). <a href="https://www.youtube.com/watch?v=XJcnxmGrxVY">Crameri too proposed a palette</a> using <code>batlow</code>, though it is only perceptually distinct for roughly the first five colors.</p>
<p>As far as I’m aware, the most thorough study that attempts to balance CVD-friendliness and aesthetics were conducted by <span class="citation" data-cites="PetroffColor2024">Petroff (2024)</span>, which features a distance metric to measure perceptual distinguishability (including with CVD), and an online survey with 22000 responses that ranks aesthetical preferences. This resulted in three palettes of six, eight, and ten colors which I am quite fond of. They are also the default used in <a href="https://lilaq.org/docs/reference/color#qualitative-color-maps">Typst’s Lilaq plotting library</a>.</p>

<div class="no-row-height column-margin column-container"><div class="">
<p><a href="../media/posts/scientific-colormaps/petroff.png" class="lightbox" data-gallery="quarto-lightbox-gallery-2"><img src="https://jianweicheong.codeberg.page/media/posts/scientific-colormaps/petroff.png" class="img-fluid"></a></p>
<p>Reprinted from <span class="citation" data-cites="PetroffColor2024">Petroff (2024)</span>.</p>
<p>Petroff also calculated the perceptual distinguishability of other common color cycles (higher is better).</p>
</div></div><div class="quarto-figure quarto-figure-center">
<figure class="figure">
<p><img src="https://jianweicheong.codeberg.page/media/posts/scientific-colormaps/colorcycles.svg" class="img-fluid figure-img"></p>
<figcaption>CVD-friendly color cycles.</figcaption>
</figure>
</div>
<p>Looking at Petroff’s analysis (see figure in the margin), there isn’t a lot of color cycles that are CVD-friendly, so it’s better to stick to Petroff’s color palettes. However, since they are typically not readily available, Okabe’s and Ito’s, and Crameri’s <code>batlowS</code> (up till the first five colors) are good alternatives.</p>
<p>Paul Tol’s “Tol Bright” is also CVD-friendly, but only for people with deuteranopia and protanopia. People with the much less common tritanopia would struggle with it<sup>5</sup>, which is why it has a low score in Petroff’s analysis.</p>
<div class="no-row-height column-margin column-container"><div id="fn5"><p><sup>5</sup>&nbsp;According to <a href="https://en.wikipedia.org/wiki/Color_blindness#Based_on_severity">Wikipedia</a> only <img src="https://latex.codecogs.com/png.latex?0.015%5C%25"> of individuals has tritanopia, which is an order of magnitude less than deuteranopia and protanopia. This is around 1000 people in the entire country of Singapore.</p></div></div></section>
<section id="summary" class="level1">
<h1>Summary</h1>
<p>While it’s true that most reputable journals will require plots to be understandable in grayscale, i.e., relying on line style/shape in line and scatter plots, or roughly ensuring the <a href="https://en.wikipedia.org/wiki/Lightness">lightness</a> ordering of colormaps, most readers are reading the colored version on a computer, and color remains an important factor and tool in scientific visualization.</p>
<p>It can emphasize, direct, focus, mislead or misrepresent scientific results. Instead of just choosing a colormap or palette based on whether it looks aesthetically pleasing to our eyes, more care and responsibility should be taken in choosing one that is perceptually uniform and linear to not mislead, and one that is CVD-friendly to be accessible to more readers.</p>



</section>


<a onclick="window.scrollTo(0, 0); return false;" id="quarto-back-to-top"><i class="bi bi-arrow-up"></i> Back to top</a><div id="quarto-appendix" class="default"><section class="quarto-appendix-contents" id="quarto-bibliography"><h2 class="anchored quarto-appendix-heading">References</h2><div id="refs" class="references csl-bib-body hanging-indent" data-entry-spacing="0">
<div id="ref-BorkinEvaluation2011" class="csl-entry">
Borkin, M., K. Gajos, A. Peters, D. Mitsouras, S. Melchionna, F. Rybicki, C. Feldman, and H. Pfister. 2011. <span>“Evaluation of Artery Visualizations for Heart Disease Diagnosis.”</span> <em>IEEE Transactions on Visualization and Computer Graphics</em> 17 (12): 2479–88. <a href="https://doi.org/10.1109/tvcg.2011.192">https://doi.org/10.1109/tvcg.2011.192</a>.
</div>
<div id="ref-CrameriThe2020" class="csl-entry">
Crameri, Fabio, Grace E. Shephard, and Philip J. Heron. 2020. <span>“The Misuse of Colour in Science Communication.”</span> <em>Nature Communications</em> 11 (1). <a href="https://doi.org/10.1038/s41467-020-19160-7">https://doi.org/10.1038/s41467-020-19160-7</a>.
</div>
<div id="ref-KovesiColour2015" class="csl-entry">
Kovesi, Peter. 2015. <span>“Good Colour Maps: How to Design Them.”</span> <a href="https://arxiv.org/abs/1509.03700">https://arxiv.org/abs/1509.03700</a>.
</div>
<div id="ref-LandiniColor2011" class="csl-entry">
Landini, Gabriel, and D Giles Perryer. 2011. <span>“More on Color Blindness.”</span> <em>Nature Methods</em> 8 (11): 891–91. <a href="https://doi.org/10.1038/nmeth.1737">https://doi.org/10.1038/nmeth.1737</a>.
</div>
<div id="ref-NunezOptimizing2018" class="csl-entry">
Nuñez, Jamie R., Christopher R. Anderton, and Ryan S. Renslow. 2018. <span>“Optimizing Colormaps with Consideration for Color Vision Deficiency to Enable Accurate Interpretation of Scientific Data.”</span> Edited by Jesús Malo. <em>PLOS ONE</em> 13 (7): e0199239. <a href="https://doi.org/10.1371/journal.pone.0199239">https://doi.org/10.1371/journal.pone.0199239</a>.
</div>
<div id="ref-PetroffColor2024" class="csl-entry">
Petroff, Matthew A. 2024. <span>“Accessible Color Sequences for Data Visualization.”</span> <a href="https://arxiv.org/abs/2107.02270">https://arxiv.org/abs/2107.02270</a>.
</div>
<div id="ref-WongPoints2011" class="csl-entry">
Wong, Bang. 2011. <span>“Points of View: Color Blindness.”</span> <em>Nature Methods</em> 8 (6): 441–41. <a href="https://doi.org/10.1038/nmeth.1618">https://doi.org/10.1038/nmeth.1618</a>.
</div>
<div id="ref-ZhouSurvey2016" class="csl-entry">
Zhou, Liang, and Charles D. Hansen. 2016. <span>“A Survey of Colormaps in Visualization.”</span> <em>IEEE Transactions on Visualization and Computer Graphics</em> 22 (8): 2051–69. <a href="https://doi.org/10.1109/tvcg.2015.2489649">https://doi.org/10.1109/tvcg.2015.2489649</a>.
</div>
</div></section></div> ]]></description>
  <category>programming</category>
  <guid>https://jianweicheong.codeberg.page/posts/colormaps.html</guid>
  <pubDate>Fri, 29 Aug 2025 16:00:00 GMT</pubDate>
</item>
<item>
  <title>The Betty Hill Star Map: Where are my aliens from?</title>
  <link>https://jianweicheong.codeberg.page/posts/betty-hill-star-map.html</link>
  <description><![CDATA[ 




<div class="callout callout-style-default callout-important callout-titled">
<div class="callout-header d-flex align-content-center">
<div class="callout-icon-container">
<i class="callout-icon"></i>
</div>
<div class="callout-title-container flex-fill">
<span class="screen-reader-only">Important</span>Disclaimer
</div>
</div>
<div class="callout-body-container callout-body">
<p>I apologize for the lack of sources provided in this post. It is quite difficult to find accessible primary sources, especially for these kinds of topics where most details are only accounted in books that are not freely available. If you do not see a source, take it with a grain of salt.</p>
</div>
</div>
<div class="page-columns page-full"><p></p><div class="no-row-height column-margin column-container"><span class="margin-aside">Here’s a Grey alien from Wikipedia. <img src="https://upload.wikimedia.org/wikipedia/commons/thumb/b/bf/Greylien.png/120px-Greylien.png" class="img-fluid" style="width:25.0%"></span></div></div>
<p>In <a href="https://en.wikipedia.org/wiki/Ufology">Ufology</a> “lore”, it is widely accepted/parroted that the well-known, short, enlarged head, with large black eyes, <a href="https://en.wikipedia.org/wiki/Grey_alien">Grey alien</a> comes from the <a href="https://en.wikipedia.org/wiki/Zeta_Reticuli">Zeta Reticuli</a> binary star system.</p>
<p>This is due to a myriad of (supposedly) independent “abductees”, “contactees”, or “insiders”, who have made mentions of Zeta Reticuli (as well as its association with the Greys) in their “lore”, e.g., Bob Lazar, Project Serpo, David Icke, Ashtar, Cosmic Awareness, or the various “Galactic Federation” “<a href="https://en.wikipedia.org/wiki/New_Age#Spirit_and_channeling">channelings</a>”<sup>1</sup>.</p>
<div class="no-row-height column-margin column-container"><div id="fn1"><p><sup>1</sup>&nbsp;I’m not going to provide sources for these, but I have around 2 decades of exposure to these stuffs and can thus speak from experience.</p></div></div><p>The very first association (as far as I know) of these Greys with the Zeta Reticuli star system originates from the <a href="https://en.wikipedia.org/wiki/Barney_and_Betty_Hill_incident">Barney and Betty Hill incident</a> back in the 1960s. Particularly, from this incident came a “star map”, crudely drawn from memory under <a href="https://en.wikipedia.org/wiki/Recovered-memory_therapy">hypnotic regression</a> 2 years after the alleged incident, which vaguely resembles a constellation of stars that include the Zeta Reticuli binary star system.</p>
<p>Today, let us examine this star map with modern tools and data, and determine if there is any credence to this association with the Zeta Reticuli star system.</p>
<p>But first, let us have some context.</p>
<section id="the-barney-and-betty-hill-incident-abridged" class="level1 page-columns page-full">
<h1>The Barney and Betty Hill Incident (abridged)</h1>
<p>On the night of 19 September 1961, American couple Barney and Betty Hill observed a light in the sky while they were driving home from vacation on some stretches of deserted roads. They stopped occasionally to observe the light with their binoculars, and eventually surmise that the light is a UFO after it descended towards their vehicle.</p>
<p>Barney observed the object with the binoculars, spotting some humanoid figures looking out of the craft’s windows. As the craft approached closer, the Hills drove away in fear of being captured. Afterward, they experienced some “missing time” where there are gaps in their memory<sup>2</sup> before they returned home.</p>
<div class="no-row-height column-margin column-container"><div id="fn2"><p><sup>2</sup>&nbsp;The experience of “missing time” is quite common among alien abduction stories.</p></div></div><p>I encourage the interested reader to read the fascinating full story themselves, but our concern here is the star map, which comes 2 years after this incident.</p>
<p>Around 2 years later, under some advice, the Hills underwent hypnosis sessions with Dr.&nbsp;Benjamin Simon to “recover” lost/repressed memories of the incident<sup>3</sup>. This was when the encounter with the UFO became an abduction story, as Barney and Betty Hill recalled (under hypnosis) that they were abducted onto the UFO, and have their memories repressed by the aliens afterward.</p>
<div class="no-row-height column-margin column-container"><div id="fn3"><p><sup>3</sup>&nbsp;Note that such <a href="https://en.wikipedia.org/wiki/Recovered-memory_therapy">recovered-memory therapy</a> are considered pseudoscience, and there is no evidence that it is effective in recovering accurate memories, much less memories repressed by alien technologies.</p></div><div id="fn4"><p><sup>4</sup>&nbsp;I guess saying something like “we are from what your people call Zeta Reticuli” was too difficult? Another story where the aliens were evasive when asked about their origin is the story of Charles Hall and the “Tall Whites”, where the Tall Whites alien responded by asking if Charles knew the name of the stars the Tall Whites used, and if not, then why does he want to know?</p></div></div><p>Betty Hill recalled (under hypnosis) that while abducted on the UFO, she asked the “leader” of the Grey-looking aliens where they came from. The leader then took out a 2D sheet depicting a “star map” (that looked 3D from the angle Betty viewed it from). Betty asked where the leader’s “home port” was, and the leader responded by asking Betty where Earth was on the map, before saying that if Betty did not know where she is then there is no point in telling her where he is from<sup>4</sup>.</p>
<p>From this hypnosis session, Betty Hill drew this “star map” that she recalled, which arguably is now the most well-known part of the story, and eventually led to the association of Zeta Reticuli with the Grey aliens.</p>
</section>
<section id="the-star-map" class="level1 page-columns page-full">
<h1>The Star Map</h1>
<div class="quarto-figure quarto-figure-center">
<figure class="figure">
<p><a href="../media/posts/betty-hill-star-map/betty_original.jpg" class="lightbox" data-gallery="quarto-lightbox-gallery-1" title="Betty Hill’s Star Map"><img src="https://jianweicheong.codeberg.page/media/posts/betty-hill-star-map/betty_original.jpg" class="img-fluid figure-img" style="width:60.0%" alt="Betty Hill’s Star Map"></a></p>
<figcaption>Betty Hill’s Star Map</figcaption>
</figure>
</div>
<p>According to the “leader” alien, the heavy lines are trade routes, the lighter lines are places they occasionally visit, and the broken (dotted) lines are expeditions.</p>
<p>One would think that given the large number of stars in the galaxy (or even in just our local stellar neighborhood), it is impossible to determine anything substantial from some arbitrarily connected dots and circles. Yet, this did not deter people from trying to interpret the star systems shown in the star map.</p>
<p>While many interpretations of the star map exist, e.g., by <a href="https://www.bibliotecapleyades.net/vida_alien/astrometrics/images/bfig11-2.gif">Charles W. Atterberg</a>, <a href="https://www.kochkyborg.de/BBHill/hill07.htm">Joachim Koch and Hans-Jürgen Kyborg</a>, or Steve Pearse, the most famous interpretation is by an elementary schoolteacher and amateur astronomer Marjorie Fish in 1969.</p>
<p>We will only examine Fish’s interpretation as it is the most popular, and is the reason for the association between the Grey aliens and the Zeta Reticuli binary star system.</p>
<section id="fishs-interpretation" class="level2 page-columns page-full">
<h2 class="anchored" data-anchor-id="fishs-interpretation">Fish’s interpretation</h2>
<div class="page-columns page-full"><p></p><div class="no-row-height column-margin column-container"><span class="margin-aside">For the convenience of the reader (click to enlarge). <a href="../media/posts/betty-hill-star-map/betty_original.jpg" class="lightbox" data-gallery="quarto-lightbox-gallery-2"><img src="https://jianweicheong.codeberg.page/media/posts/betty-hill-star-map/betty_original.jpg" class="img-fluid"></a></span></div></div>
<div class="quarto-figure quarto-figure-center">
<figure class="figure">
<p><a href="../media/posts/betty-hill-star-map/fishmap_better2.jpg" class="lightbox" data-gallery="quarto-lightbox-gallery-3" title="Marjorie Fish’s intepretation of the star map using data from the Gliese catalog."><img src="https://jianweicheong.codeberg.page/media/posts/betty-hill-star-map/fishmap_better2.jpg" class="img-fluid figure-img" alt="Marjorie Fish’s intepretation of the star map using data from the Gliese catalog."></a></p>
<figcaption>Marjorie Fish’s intepretation of the star map using data from the <a href="https://en.wikipedia.org/wiki/Gliese_Catalogue_of_Nearby_Stars">Gliese catalog</a>.</figcaption>
</figure>
</div>
<p>Admittedly, it looks quite similar to Betty Hill’s star map, and even more impressive is that this interpretation was derived from physical models of nearby stars (assuming our Sun is in the star map) that Fish herself built using data from the Gliese catalog.</p>
<p>Due to the large numbers of stars, <a href="https://www.nicap.org/reports/hillmap.htm">Fish filtered out stars based on exobiological considerations</a>. It took years but she succeeded in finding a constellation of stars and vantage point that resembles Betty Hill’s star map using her homemade model.</p>
<p>There are a lot of differences however, e.g., the line to Alpha Mensae is much longer than in Betty Hill’s star map, while the line to Gliese 86 is much shorter. The distance between Tau Ceti and 82 Eridani is also much shorter than expected. All in all, it is not a perfect match.</p>
<p>Despite that, Fish’s interpretation became so popular that it was even <a href="https://www.youtube.com/watch?v=YEaucytiEwM">discussed by Carl Sagan on <em>Cosmos</em></a><sup>5</sup>. Sagan’s argument is that if one can pick and choose from a large number of stars from any vantage point, then one can always find a constellation of stars that resembles Betty Hill’s star map<sup>6</sup>.</p>
<div class="no-row-height column-margin column-container"><div id="fn5"><p><sup>5</sup>&nbsp;Sagan demonstrated that by removing the lines, there are hardly any resemblance between the two maps at all. <a href="../media/posts/betty-hill-star-map/sagan_comparison.png" class="lightbox" data-gallery="quarto-lightbox-gallery-4"><img src="https://jianweicheong.codeberg.page/media/posts/betty-hill-star-map/sagan_comparison.png" class="img-fluid"></a></p></div><div id="fn6"><p><sup>6</sup>&nbsp;More on this later as we attempt to try this out for ourselves.</p></div></div><p>Nevertheless, the most significant result of Fish’s interpretation is that the two big circles (often thought of as the home systems of the aliens) turns out to be the Zeta Reticuli binary star system. This is what sparked the beginning of the association between Grey-type aliens and Zeta Reticuli.</p>
<div class="callout callout-style-default callout-warning callout-titled">
<div class="callout-header d-flex align-content-center collapsed" data-bs-toggle="collapse" data-bs-target=".callout-2-contents" aria-controls="callout-2" aria-expanded="false" aria-label="Toggle callout">
<div class="callout-icon-container">
<i class="callout-icon"></i>
</div>
<div class="callout-title-container flex-fill">
<span class="screen-reader-only">Warning</span>Misleading visuals of Fish’s interpretation
</div>
<div class="callout-btn-toggle d-inline-block border-0 py-1 ps-1 pe-0 float-end"><i class="callout-toggle"></i></div>
</div>
<div id="callout-2" class="callout-2-contents callout-collapse collapse">
<div class="callout-body-container callout-body">
<p>The reader should take note that there are various misleading visualizations of the Fish’s interpretation that are often being shared online. These misleading visuals exaggerate the similarity between Fish’s interpretation and Betty Hill’s star map. Some examples are the one on Wikipedia:</p>
<p><a href="../media/posts/betty-hill-star-map/fishmap_misleading1.png" class="lightbox" data-gallery="quarto-lightbox-gallery-5"><img src="https://jianweicheong.codeberg.page/media/posts/betty-hill-star-map/fishmap_misleading1.png" class="img-fluid" style="width:65.0%"></a></p>
<p>and this one that is often shared on social medias that is just blatantly misleading:</p>
<p><a href="../media/posts/betty-hill-star-map/fishmap_misleading2.jpg" class="lightbox" data-gallery="quarto-lightbox-gallery-6"><img src="https://jianweicheong.codeberg.page/media/posts/betty-hill-star-map/fishmap_misleading2.jpg" class="img-fluid" style="width:50.0%"></a></p>
</div>
</div>
</div>
</section>
<section id="reproducing-fishs-interpretation" class="level2 page-columns page-full">
<h2 class="anchored" data-anchor-id="reproducing-fishs-interpretation">Reproducing Fish’s Interpretation</h2>
<p>Let us attempt to reproduce Fish’s results by using the <a href="https://en.wikipedia.org/wiki/Hipparcos">Hipparcos catalog</a> from the 1990s<sup>7</sup>, which contains much more precise data than the Gliese catalog that Fish used in 1969. Specifically, we will create a 3D model of nearby stars within 30 parsecs of our Solar System.</p>
<div class="no-row-height column-margin column-container"><div id="fn7"><p><sup>7</sup>&nbsp;The Gaia catalog was considered, but it excludes a number of bright stars and the star designations are too long and cumbersome to work with.</p></div></div><p>The lines of codes are quite short, and I encourage the interested reader who wants their own 3D model to follow along, especially if one has experience in Python.</p>
<section id="getting-the-positions-of-nearby-stars" class="level3">
<h3 class="anchored" data-anchor-id="getting-the-positions-of-nearby-stars">Getting the positions of nearby stars</h3>
<p>We can easily retrieve the catalog as a <a href="https://pandas.pydata.org/"><em>pandas</em></a> dataframe in Python with the <a href="https://rhodesmill.org/skyfield/"><em>Skyfield</em></a> package.</p>
<div class="callout callout-style-simple callout-note no-icon callout-titled">
<div class="callout-header d-flex align-content-center" data-bs-toggle="collapse" data-bs-target=".callout-3-contents" aria-controls="callout-3" aria-expanded="true" aria-label="Toggle callout">
<div class="callout-icon-container">
<i class="callout-icon no-icon"></i>
</div>
<div class="callout-title-container flex-fill">
<span class="screen-reader-only">Note</span>Skip this if you are not interested.
</div>
<div class="callout-btn-toggle d-inline-block border-0 py-1 ps-1 pe-0 float-end"><i class="callout-toggle"></i></div>
</div>
<div id="callout-3" class="callout-3-contents callout-collapse collapse show">
<div class="callout-body-container callout-body">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb1" style="background: #f1f3f5;"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb1-1"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">import</span> numpy <span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">as</span> np</span>
<span id="cb1-2"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">import</span> pandas <span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">as</span> pd</span>
<span id="cb1-3"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">from</span> skyfield.api <span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">import</span> Star, load</span>
<span id="cb1-4"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">from</span> skyfield.data <span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">import</span> hipparcos</span>
<span id="cb1-5"></span>
<span id="cb1-6"><span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">with</span> load.<span class="bu" style="color: null;
background-color: null;
font-style: inherit;">open</span>(hipparcos.URL) <span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">as</span> f:</span>
<span id="cb1-7">    df <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> hipparcos.load_dataframe(f)</span></code></pre></div></div>
<p>To get the positions of nearby stars relative to our Sun, we first retrieve the <a href="https://en.wikipedia.org/wiki/Jet_Propulsion_Laboratory_Development_Ephemeris">JPL ephemeris DE421</a>, which contains the positions of major bodies in the Solar System from 1900–2050, before finding the Sun’s position relative to the Solar System’s barycenter on 19 September 1961.</p>
<p>These are actually mostly unneeded information, but <em>Skyfield</em> requires a <em>Barycentric</em> object to retrieve the observed position of stars with respect to that object’s position at a certain time, and we will have to make do since we want <em>Skyfield</em> to do the calculations for us.</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb2" style="background: #f1f3f5;"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb2-1">eph <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> load(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"de421.bsp"</span>)</span>
<span id="cb2-2">sun <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> eph[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"sun"</span>]</span>
<span id="cb2-3"></span>
<span id="cb2-4">ts <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> load.timescale()</span>
<span id="cb2-5">t <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> ts.utc(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1961</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">9</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">20</span>)</span>
<span id="cb2-6">sun_barycentric <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> sun.at(t)</span></code></pre></div></div>
<p>Next, we let <em>Skyfield</em> calculate the observed position (in terms of <img src="https://latex.codecogs.com/png.latex?x">, <img src="https://latex.codecogs.com/png.latex?y">, <img src="https://latex.codecogs.com/png.latex?z"> coordinates and distance relative to the Sun) of each star in the dataframe of the catalog, before storing them in their corresponding list. We also convert the units from <a href="https://en.wikipedia.org/wiki/Astronomical_unit">AU (astronomical unit)</a> to <a href="https://en.wikipedia.org/wiki/Parsec">parsec</a> as it’s easier to work with. Note that <code>hips</code> contains each star’s designation in the catalog, e.g., the star Sirius has the designation HIP 32349.</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb3" style="background: #f1f3f5;"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb3-1">au_1_parsec <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">206265</span></span>
<span id="cb3-2"></span>
<span id="cb3-3">distances_pc <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> []</span>
<span id="cb3-4">coords_pc_x <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> []</span>
<span id="cb3-5">coords_pc_y <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> []</span>
<span id="cb3-6">coords_pc_z <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> []</span>
<span id="cb3-7">hips <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> []</span>
<span id="cb3-8"><span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">for</span> i, row <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">in</span> df.iterrows():</span>
<span id="cb3-9">    star <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> Star.from_dataframe(row)</span>
<span id="cb3-10">    star_astrometric <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> sun_barycentric.observe(star)</span>
<span id="cb3-11">    _, _, distance_au <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> star_astrometric.radec()</span>
<span id="cb3-12">    coords_au <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> star_astrometric.xyz</span>
<span id="cb3-13"></span>
<span id="cb3-14">    distances_pc.append(distance_au.au <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">/</span> au_1_parsec)</span>
<span id="cb3-15">    coords_pc <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> coords_au.au <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">/</span> au_1_parsec</span>
<span id="cb3-16">    coords_pc_x.append(coords_pc[<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>])</span>
<span id="cb3-17">    coords_pc_y.append(coords_pc[<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>])</span>
<span id="cb3-18">    coords_pc_z.append(coords_pc[<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>])</span>
<span id="cb3-19">    hips.append(i)</span></code></pre></div></div>
<p>Finally, we store these data for each star in a new dataframe, filtered it such that we only keep stars within 30 parsecs (~100 light years), leaving us with just 2350 stars, before saving it to a CSV file.</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb4" style="background: #f1f3f5;"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb4-1">df_new <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> pd.DataFrame(</span>
<span id="cb4-2">    np.array([hips, coords_pc_x, coords_pc_y, coords_pc_z, distances_pc]).T,</span>
<span id="cb4-3">    columns<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span>[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"hip"</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"coords_x"</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"coords_y"</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"coords_z"</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"distances"</span>],</span>
<span id="cb4-4">)</span>
<span id="cb4-5"></span>
<span id="cb4-6">df_near <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> df_new[df_new[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"distances"</span>] <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">30</span>]</span>
<span id="cb4-7">df_near.to_csv(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"30pc_stars.csv"</span>, index<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">False</span>)</span></code></pre></div></div>
</div>
</div>
</div>
<p>Note that the obtained positions are the <strong>observed positions</strong> from our Solar System, that is, the stars actual positions would have changed due to the time it takes for light from these stars to reach us, and an alien star map would probably use the latter instead of the observed positions, let alone <em>our</em> observed positions. However, the difference are negligible and this save us some calculations, so we will stick with this.</p>
</section>
<section id="plotting-the-nearby-stars" class="level3 page-columns page-full">
<h3 class="anchored" data-anchor-id="plotting-the-nearby-stars">Plotting the nearby stars</h3>
<p>Next, we will plot these 2350 stars (+ the Sun) in <a href="https://www.blender.org/download/">Blender</a>. Blender is chosen because it’s fun, and it’s easy to control and animate the camera when we are looking for different vantage points. The hybrid graphical and Python workflow also greatly makes our lives easier as Python can programmatically handle the placing of the 2350 stars at the right position, while I can easily change the looks of the stars in Blender’s GUI<sup>8</sup>.</p>
<div class="no-row-height column-margin column-container"><div id="fn8"><p><sup>8</sup>&nbsp;One can also use programs like <a href="https://gaiasky.space/">Gaia Sky</a> or <a href="https://spaceengine.org/">Space Engine</a> to do their own investigation.</p></div></div><div class="callout callout-style-simple callout-note no-icon callout-titled">
<div class="callout-header d-flex align-content-center" data-bs-toggle="collapse" data-bs-target=".callout-4-contents" aria-controls="callout-4" aria-expanded="true" aria-label="Toggle callout">
<div class="callout-icon-container">
<i class="callout-icon no-icon"></i>
</div>
<div class="callout-title-container flex-fill">
<span class="screen-reader-only">Note</span>Skip this if you are not interested
</div>
<div class="callout-btn-toggle d-inline-block border-0 py-1 ps-1 pe-0 float-end"><i class="callout-toggle"></i></div>
</div>
<div id="callout-4" class="callout-4-contents callout-collapse collapse show">
<div class="callout-body-container callout-body">
<p>First, let us load the CSV file that we saved earlier and store them in the list <code>stars</code>.</p>
<p>It’s quite troublesome to install external packages to Blender’s builtin Python, and so we used the builtin <code>csv</code> package instead of <code>pandas</code> dataframe this time.</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb5" style="background: #f1f3f5;"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb5-1"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">import</span> bpy</span>
<span id="cb5-2"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">import</span> csv</span>
<span id="cb5-3"></span>
<span id="cb5-4"><span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">with</span> <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">open</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"30pc_stars.csv"</span>) <span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">as</span> f:</span>
<span id="cb5-5">    reader <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> csv.DictReader(f, delimiter<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">","</span>)</span>
<span id="cb5-6"></span>
<span id="cb5-7">    stars <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> []</span>
<span id="cb5-8">    <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">for</span> entry <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">in</span> reader:</span>
<span id="cb5-9">        stars.append(entry)</span></code></pre></div></div>
<p>In Blender, I have prepared different <em>materials</em> for the Sun (<code>sun</code>), stars in Fish’s interpretations (<code>starmap</code>), and the rest of the stars (<code>starlight</code>). We can refer to these materials in Python as follows.</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb6" style="background: #f1f3f5;"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb6-1">sun_mat <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> bpy.data.materials.get(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"sun"</span>)</span>
<span id="cb6-2">starmap_mat <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> bpy.data.materials.get(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"starmap"</span>)</span>
<span id="cb6-3">starlight_mat <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> bpy.data.materials.get(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"starlight"</span>)</span></code></pre></div></div>
<p>I also made a helper function to easily place stars as <em>UV spheres</em>.</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb7" style="background: #f1f3f5;"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb7-1"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">def</span> uv_sphere_add(radius<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>, name<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">None</span>, location<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span>(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>)):</span>
<span id="cb7-2">    bpy.ops.mesh.primitive_uv_sphere_add(radius<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span>radius, location<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span>location)</span>
<span id="cb7-3">    obj <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> bpy.context.<span class="bu" style="color: null;
background-color: null;
font-style: inherit;">object</span></span>
<span id="cb7-4">    <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">if</span> name:</span>
<span id="cb7-5">        obj.name <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> name</span>
<span id="cb7-6">    <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">return</span> obj</span></code></pre></div></div>
<p>Creating the Sun and adding the <code>sun</code> material to it is as simple as</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb8" style="background: #f1f3f5;"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb8-1">sun_obj <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> uv_sphere_add(<span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0.2</span>, name<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"sun"</span>, location<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span>(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>))</span>
<span id="cb8-2">sun_obj.data.materials.append(sun_mat)</span></code></pre></div></div>
<p>We can now repeat the process for each star. By checking the HIP designations of the stars in Fish’s interpretation, we can change the materials of these stars to the <code>starmap</code> material, while keeping the <code>starlight</code> material for the rest of the stars. Also note that we scaled the positions up by a factor of 10, i.e., 1 parsec correspond to 10 units of distance in Blender.</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb9" style="background: #f1f3f5;"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb9-1">fish_starmap <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> [</span>
<span id="cb9-2">    <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"15330.0"</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"15371.0"</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"29271.0"</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"15510.0"</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"10138.0"</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"8102.0"</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"7981.0"</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"3093.0"</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"7918.0"</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"12843.0"</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"7235.0"</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"11072.0"</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"10798.0"</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"10164.0"</span>,</span>
<span id="cb9-3">]</span>
<span id="cb9-4"><span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">for</span> star <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">in</span> stars:</span>
<span id="cb9-5">    star_obj <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> uv_sphere_add(</span>
<span id="cb9-6">        <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0.2</span>,</span>
<span id="cb9-7">        name<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span>star[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"hip"</span>],</span>
<span id="cb9-8">        location<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span>(</span>
<span id="cb9-9">            <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">float</span>(star[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"coords_x"</span>]) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">10</span>,</span>
<span id="cb9-10">            <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">float</span>(star[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"coords_y"</span>]) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">10</span>,</span>
<span id="cb9-11">            <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">float</span>(star[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"coords_z"</span>]) <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">10</span>,</span>
<span id="cb9-12">        ),</span>
<span id="cb9-13">    )</span>
<span id="cb9-14">    <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">if</span> star[<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"hip"</span>] <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">in</span> fish_starmap:</span>
<span id="cb9-15">        star_obj.data.materials.append(starmap_mat)</span>
<span id="cb9-16">    <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">else</span>:</span>
<span id="cb9-17">        star_obj.data.materials.append(starlight_mat)</span></code></pre></div></div>
</div>
</div>
</div>
<p>There we have it, we now have a 3D model of 2350 stars within 30 parsecs of our Sun.</p>
<div class="quarto-figure quarto-figure-center">
<figure class="figure">
<p><video src="../media/posts/betty-hill-star-map/30pc_spin_fade_connections.webm" class="img-fluid" loop="true" autoplay="true" controls=""><a href="../media/posts/betty-hill-star-map/30pc_spin_fade_connections.webm">Video</a></video></p>
<figcaption>Animation of stars within 30 parsecs, with stars from Fish’s interpretation colored red, and our Sun colored yellow.</figcaption>
</figure>
</div>
<p>After connecting the stars accordingly, we can find a vantage point that roughly matches with Fish’s interpretation.</p>
<p><video src="../media/posts/betty-hill-star-map/fish_map_animation.webm" class="img-fluid" loop="true" autoplay="true" controls=""><a href="../media/posts/betty-hill-star-map/fish_map_animation.webm">Video</a></video></p>
<div class="page-columns page-full"><p></p><div class="no-row-height column-margin column-container"><span class="margin-aside">For the convenience of the reader (click to enlarge). <a href="../media/posts/betty-hill-star-map/fishmap_better2.jpg" class="lightbox" data-gallery="quarto-lightbox-gallery-7"><img src="https://jianweicheong.codeberg.page/media/posts/betty-hill-star-map/fishmap_better2.jpg" class="img-fluid"></a><a href="../media/posts/betty-hill-star-map/betty_original.jpg" class="lightbox" data-gallery="quarto-lightbox-gallery-8"><img src="https://jianweicheong.codeberg.page/media/posts/betty-hill-star-map/betty_original.jpg" class="img-fluid"></a></span></div></div>
<div class="quarto-figure quarto-figure-center">
<figure class="figure">
<p><a href="../media/posts/betty-hill-star-map/fishmap.jpg" class="lightbox" data-gallery="quarto-lightbox-gallery-9" title="3D reproduction of Fish’s interpretation of Betty Hill’s star map."><img src="https://jianweicheong.codeberg.page/media/posts/betty-hill-star-map/fishmap.jpg" class="img-fluid figure-img" alt="3D reproduction of Fish’s interpretation of Betty Hill’s star map."></a></p>
<figcaption>3D reproduction of Fish’s interpretation of Betty Hill’s star map.</figcaption>
</figure>
</div>
<p>First of all, note that Kappa Fornacis is way off. This is due to the more precise data provided by the Hipparcos catalog as compared to the Gliese catalog that Marjorie Fish used. Likewise, Gliese 86.1 is actually nowhere close at almost 200 light years away and thus is missing.</p>
<p>These discrepancies were most likely also acknowledged by Marjorie Fish as new data came out<sup>9</sup>, and might be a deal-breaker as <a href="https://www.nicap.org/articles/hillzeta.htm">according to Fish’s account of her meeting with Betty Hill, Hill claimed to remember the distinctive triangle formation to the left in addition to the ones connected by lines</a>.</p>
<div class="no-row-height column-margin column-container"><div id="fn9"><p><sup>9</sup>&nbsp;Not that it matters, but according to her niece, <a href="https://badufos.blogspot.com/2013/07/rip-marjorie-fish-and-her-star-map.html?showComment=1377491596140#c4776630570236524431">Fish did not actually repudiate her star map</a>, and that her <a href="https://www.legacy.com/us/obituaries/portclintonnewsherald/name/marjorie-fish-obituary?id=15022043">obituary saying so</a> was in error.</p></div></div><p>Nevertheless, huge kudos to Marjorie Fish for producing her version of the star map by hand to such a great accuracy.</p>
</section>
</section>
</section>
<section id="is-the-star-map-even-useful" class="level1 page-columns page-full">
<h1>Is the Star Map even useful?</h1>
<p>As mentioned earlier, Carl Sagan argued that if one can pick and choose from a large number of stars and from any vantage point, one can always find a constellation of stars that resembles Betty Hill’s star map.</p>
<p>I have no doubt that we can indeed find our own star map that resembles Betty Hill’s from our stellar neighborhood (in fact I’ll present one later), but Marjorie Fish achieved this with less than 50 stars.</p>
<p>Is this possible?</p>
<section id="randomly-generated-stars" class="level2 page-columns page-full">
<h2 class="anchored" data-anchor-id="randomly-generated-stars">Randomly generated stars</h2>
<p>Here, I randomly generated 50 stars with uniformly distributed positions, and attempt to find a vantage point and constellation that resembles Betty Hill’s star map.</p>
<p>I assure you that there is no cherry-picking and I’m not just drawing lines on a 2D projection of the 50 stars either, but instead ensured that the star map made physical sense in terms of distances<sup>10</sup>.</p>
<div class="no-row-height column-margin column-container"><div id="fn10"><p><sup>10</sup>&nbsp;Which is why it might look like there are better solutions that I didn’t take, but those solutions only work as a 2D projection from this vantage point, and not in terms of physical distance.</p></div></div><div class="quarto-figure quarto-figure-center">
<figure class="figure">
<p><a href="../media/posts/betty-hill-star-map/starmap_randoms.png" class="lightbox" data-gallery="quarto-lightbox-gallery-10" title="Three star maps that somewhat resembles Betty Hill’s star map, created from three different sets of randomly generated 50 star positions."><img src="https://jianweicheong.codeberg.page/media/posts/betty-hill-star-map/starmap_randoms.png" class="img-fluid figure-img" alt="Three star maps that somewhat resembles Betty Hill’s star map, created from three different sets of randomly generated 50 star positions."></a></p>
<figcaption>Three star maps that somewhat resembles Betty Hill’s star map, created from three different sets of randomly generated 50 star positions.</figcaption>
</figure>
</div>
<p>As you can see, it seems that it’s not that difficult to find a constellation of stars that resembles Betty Hill’s star map from just 50 stars, even when these stars are not based on reality. In fact, each one took me just around five minutes.</p>
<p>These are of course not perfect matches (but neither are Fish’s, Atterberg’s, or Koch-Kyborg’s), and there might be better solutions if given enough time (or using some algorithmic method).</p>
</section>
<section id="a-new-interpretation-of-the-star-map" class="level2 page-columns page-full">
<h2 class="anchored" data-anchor-id="a-new-interpretation-of-the-star-map">A new interpretation of the star map</h2>
<p>Finally, let me present an alternate interpretation using just star systems in our local neighborhood of up to 22 light years (~7 parsecs) away from us, much closer than Fish’s and Atterberg’s interpretations.</p>
<p><video src="../media/posts/betty-hill-star-map/mine2_animation.webm" class="img-fluid" loop="true" autoplay="true" controls=""><a href="../media/posts/betty-hill-star-map/mine2_animation.webm">Video</a></video></p>
<div class="page-columns page-full"><p></p><div class="no-row-height column-margin column-container"><span class="margin-aside">For the convenience of the reader (click to enlarge). <a href="../media/posts/betty-hill-star-map/fishmap.jpg" class="lightbox" data-gallery="quarto-lightbox-gallery-11"><img src="https://jianweicheong.codeberg.page/media/posts/betty-hill-star-map/fishmap.jpg" class="img-fluid"></a><a href="../media/posts/betty-hill-star-map/betty_original.jpg" class="lightbox" data-gallery="quarto-lightbox-gallery-12"><img src="https://jianweicheong.codeberg.page/media/posts/betty-hill-star-map/betty_original.jpg" class="img-fluid"></a></span></div></div>
<div class="quarto-figure quarto-figure-center">
<figure class="figure">
<p><a href="../media/posts/betty-hill-star-map/mine2_starmap.png" class="lightbox" data-gallery="quarto-lightbox-gallery-13" title="My interpretation of Betty Hill’s star map."><img src="https://jianweicheong.codeberg.page/media/posts/betty-hill-star-map/mine2_starmap.png" class="img-fluid figure-img" alt="My interpretation of Betty Hill’s star map."></a></p>
<figcaption>My interpretation of Betty Hill’s star map.</figcaption>
</figure>
</div>
<p>It’s not a perfect match, but again I emphasize that neither are Fish’s, Atterberg’s, or Koch-Kyborg’s.</p>
<p>It’s not difficult to find this constellation of stars either:</p>
<blockquote class="blockquote">
<p>Just by looking at prominent stars in the vicinity of our Solar System, I realized that the Bernard’s star, Proxima/Alpha Centuri, our Solar System, and Sirius, forms the “V” shape formation that appears on Betty Hill’s star map.</p>
<p>I then looked for stars in the “foreground” of this “V” shape formation and found Tau Ceti (and YZ Ceti). Quite coincidentally, slightly in front of Tau Ceti (and YZ Ceti) is GJ 1005, a binary star system, exactly what we need to match with Betty Hill’s star map.</p>
<p>82 G. Eridani seems to be the only possible candidate to the right of GJ 1005, and Van Maanen 2 is right above Tau Ceti (and YZ Ceti). It was then either Tau Ceti or YZ Ceti, Gliese 876 or HIP 114110, HR 7703 or Gliese 784, and Gliese 829 or Altair. I simply chose the ones that best matches with Betty Hill’s star map.</p>
</blockquote>
<p>In my opinion, my interpretation matches better with Betty Hill’s star map than Fish’s in certain parts. I even got the triangle formation on the left (by chance really, it was an afterthought)<sup>11</sup>. Furthermore, the vantage point is facing towards the galactic center, and is slightly above the galactic plane looking downwards, in other words it is not as arbitrary. A number of these star systems also have (or might have) exoplanets.</p>
<div class="no-row-height column-margin column-container"><div id="fn11"><p><sup>11</sup>&nbsp;If you are concerned about the lone unconnected star at the middle top part of Betty Hill’s star map, Psi Velorum is the only star that appears around that location if we filter out stars with absolute magnitude <img src="https://latex.codecogs.com/png.latex?%3E%204">.</p></div></div><p>The distances and lines drawn also make sense. For example, in Fish’s interpretation, one might wonder why the aliens do not connect 82 G. Eridani or Tau Ceti to Sol/our Sun given that they are right beside each other. In my interpretation, Van Maanen 2 is actually in a different direction to Tau Ceti (it is above Tau Ceti and GJ 1005, you can see it in the video above), and so it’s not surprising that the aliens drew the lines to them separately.</p>
<p>This interpretation however, would place the aliens’ origin at <a href="https://en.wikipedia.org/wiki/GJ_1005">GJ 1005</a> (or Gliese 1005), a binary red dwarf system<sup>12</sup>. As far as I am aware, there are no candidate exoplanets on GJ 1005 as of the writing of this post, but it is not uncommon for red dwarfs to have some.</p>
<div class="no-row-height column-margin column-container"><div id="fn12"><p><sup>12</sup>&nbsp;Finally somewhere more mundane. I mean why do all the aliens in Ufology lore always have to coincidentally come from some named star systems?</p></div></div><p>I also did not account for how likely it is for an alien species to visit these star systems, e.g., I didn’t check if they are variable stars or close binaries or what not which Fish excluded. However, these stars are all quite close to each other, and there aren’t many stars in the vicinity of these routes for the aliens to “skip” or be “uninterested by” like in Fish’s interpretation.</p>
<p><strong>It is important to note that the point here is not to present what I think is the “correct” interpretation of Betty Hill’s star map. Instead, the point is that one can easily find multiple interpretations that resembles it.</strong></p>
<p>In other words, it’s pointless to examine the star map to try to figure out where the aliens come from.</p>
</section>
</section>
<section id="conclusion" class="level1 page-columns page-full">
<h1>Conclusion</h1>
<p>In conclusion, even if we were to be extremely generous and accept that:</p>
<ol type="1">
<li>The UFO that the Hills encountered is an alien craft.</li>
<li>The Hills were abducted by the aliens.</li>
<li>Hypnotic regression is reliable in recovering these abduction memories that were repressed by the aliens.</li>
<li>Betty Hill remembered the star map to a reasonable accuracy while under hypnosis.</li>
<li>Betty Hill (who claimed she’s not good with perspective) drew the star map to a reasonable accuracy while under hypnosis.</li>
</ol>
<p><strong>We still cannot conclude anything from the star map given the large number of possible different interpretations it can lead to.</strong></p>
<p>Carl Sagan was right that if one can pick and choose from a large number of stars from any vantage point, then one can always find a constellation of stars that resembles the star map. We demonstrated this for randomly generated stars, and provided an example using stars in our local stellar neighborhood.</p>
<p>The fact that Fish, Atterberg, Koch and Kyborg<sup>13</sup>, and me, each have our own interpretation of the star map further supports this argument. While Fish only considered a small subset of stars that are Sun-like, we now know that the many red dwarfs in our local stellar neighborhood can host exoplanets, and thus greatly increase the number of possible interpretations.</p>
<div class="no-row-height column-margin column-container"><div id="fn13"><p><sup>13</sup>&nbsp;The Koch-Kyborg interpretation actually placed the star map in our Solar System, i.e., the objects in the map are our planets and some large asteroids, but my point still stands.</p></div></div><p>But what’s with the association between Grey aliens and the Zeta Reticuli star system from the various (supposedly) independent sources then? Especially since the Fish’s interpretation failed to reproduce the triangle formation on the left of Betty Hill’s star map.</p>
<p>I think there are 3 possible explanations:</p>
<ol type="1">
<li><p>These sources are just parroting each other. A lot of these “independent” sources share the same “lore”, e.g., in addition to Greys from Zeta Reticuli, they also have <a href="https://en.wikipedia.org/wiki/Reptilian_conspiracy_theory">Reptilians from Alpha Draconis</a>, taller Greys from Orion, and human-looking <a href="https://en.wikipedia.org/wiki/Nordic_aliens">Nordics</a> or Tall Whites from the Pleiades, Lyra, Arcturus, etc.<sup>14</sup></p></li>
<li><p>The Grey aliens are really from Zeta Reticuli, and it’s a remarkable coincidence that Fish’s interpretation of the star map shows Zeta Reticuli as the home system of the Hills’ abductors.</p></li>
<li><p>These aliens are conducting a psyop on us to hide their real origins. Recall that the “leader” alien who abducted the Hills, as well as the “Tall White” alien of the Charles Hall story, seems evasive and dodged the question when asked about their origins (I’m only half joking here).</p></li>
</ol>
<div class="no-row-height column-margin column-container"><div id="fn14"><p><sup>14</sup>&nbsp;Isn’t it a little sus that not only are these aliens human-looking, they are also often described to be attractive by our 21st century standards?</p></div></div><p>Anyway, this is just a fun excuse for me to play with Blender, and I am by no means a UFO investigator, nor am I extremely well-read into this story or Ufology “lore”.</p>
<p>Though, considering that there were multiple inconsistencies with the aliens in Betty’s account of the abduction while under hypnosis, e.g., the aliens not knowing what “older”, “age”, and even “yellow” meant despite being able to speak English and communicate telepathically, my opinion is that the recovered memories from hypnotic regression are most likely confabulations or false memories<sup>15</sup>. I believe false memories were also what Dr.&nbsp;Simon concluded after the multiple hypnosis sessions he conducted with the Hills.</p>


<div class="no-row-height column-margin column-container"><div id="fn15"><p><sup>15</sup>&nbsp;Again, I remind that such methods were never proven to be reliable in recovering lost or repressed memories.</p></div></div></section>


<a onclick="window.scrollTo(0, 0); return false;" id="quarto-back-to-top"><i class="bi bi-arrow-up"></i> Back to top</a> ]]></description>
  <category>programming</category>
  <category>fun</category>
  <category>fringe</category>
  <guid>https://jianweicheong.codeberg.page/posts/betty-hill-star-map.html</guid>
  <pubDate>Wed, 16 Jul 2025 16:00:00 GMT</pubDate>
</item>
<item>
  <title>Useful tools for the academic</title>
  <link>https://jianweicheong.codeberg.page/posts/tools-academic.html</link>
  <description><![CDATA[ 




<p>Throughout my time in academia, there are many instances where I witnessed the quickness of my colleagues in picking up and gaining proficiency in some new skills or tools that serves to enhance their work.</p>
<p>These not only include new domain knowledge, e.g., learning machine learning to solve a niche problem, but also tools that aids in their work. The regnant <a href="https://en.wikipedia.org/wiki/LaTeX">LaTeX</a> needs no introduction, and picking up C/C++ to speed up computation or learning <a href="https://en.wikipedia.org/wiki/Git">Git</a> for collaborative works are quite common.</p>
<p>However, it seems to me that many are satisfied in their “local minima”, and do not have the time or effort to look for new tools that might enhance their work. Therefore, here I introduce some tools that I found to be useful that I think more academics should use<sup>1</sup>.</p>
<div class="no-row-height column-margin column-container"><div id="fn1"><p><sup>1</sup>&nbsp;This list might be updated as more tools prove themselves useful.</p></div></div><section id="pandoc" class="level2 page-columns page-full">
<h2 class="anchored" data-anchor-id="pandoc">Pandoc</h2>
<p><a href="https://pandoc.org/">Pandoc</a> is a <em>universal document converter</em>. It can convert MS Word to and from LaTeX, an HTML page to EPUB, or MS PowerPoint to HTML, and many more! However, the most popular feature of the universal converter is the ability to write in <a href="https://en.wikipedia.org/wiki/Markdown">Markdown</a> and convert it to a document or a presentation slide.</p>
<p>Here, we write in simple Markdown and convert to a PDF document via LaTeX<sup>2</sup>.</p>
<div class="no-row-height column-margin column-container"><div id="fn2"><p><sup>2</sup>&nbsp;We can also convert to presentation slides via LaTeX beamer (or <a href="https://revealjs.com/">reveal.js</a>): <a href="../media/posts/tools-academic/pandoc_beamer.png" class="lightbox" data-gallery="quarto-lightbox-gallery-1"><img src="https://jianweicheong.codeberg.page/media/posts/tools-academic/pandoc_beamer.png" class="img-fluid"></a></p></div></div><div class="quarto-figure quarto-figure-center">
<figure class="figure">
<p><a href="../media/posts/tools-academic/pandoc.png" class="lightbox" data-gallery="quarto-lightbox-gallery-2" title="Markdown is much easier to write as compared to LaTeX."><img src="https://jianweicheong.codeberg.page/media/posts/tools-academic/pandoc.png" class="img-fluid figure-img" alt="Markdown is much easier to write as compared to LaTeX."></a></p>
<figcaption>Markdown is much easier to write as compared to LaTeX.</figcaption>
</figure>
</div>
<p>For a long time I have used Pandoc for all my write-ups, notes, and presentation slides due to the simplicity of writing Markdown over LaTeX, MS Word, and MS PowerPoint<sup>3</sup>, but I have since moved to better tools below.</p>
<div class="no-row-height column-margin column-container"><div id="fn3"><p><sup>3</sup>&nbsp;Combined with wrangling data in Python/Julia/R over MS Excel, I have completely ditched MS Office.</p></div></div></section>
<section id="quarto" class="level2 page-columns page-full">
<h2 class="anchored" data-anchor-id="quarto">Quarto</h2>
<p>Coming from the lovely R community and building on top of Pandoc, <a href="https://quarto.org/">Quarto</a> is the successor to <a href="https://rmarkdown.rstudio.com/">R Markdown</a>, an attempt at <a href="https://en.wikipedia.org/wiki/Literate_programming">literate programming</a>. R Markdown does the same as Pandoc, allowing one to write in Markdown and outputs to documents and presentations etc., with the additional feature that it also allows one to run and output R codes inside the document. Quarto extends this functionality to include Python, Julia, and Observable in addition to R.</p>
<div class="quarto-figure quarto-figure-center">
<figure class="figure">
<p><img src="https://jianweicheong.codeberg.page/media/posts/tools-academic/qmd-how-it-works.png" class="img-fluid figure-img"></p>
<figcaption>Reprinted from <a href="https://quarto.org/docs/get-started/hello/vscode.html#how-it-works">Quarto’s tutorial</a>.</figcaption>
</figure>
</div>
<p>This functionality allows us to mix computation with its documentations, explanations, or mathematics in the same document.</p>
<div class="quarto-figure quarto-figure-center">
<figure class="figure">
<p><a href="../media/posts/tools-academic/quarto.png" class="lightbox" data-gallery="quarto-lightbox-gallery-3" title="The output of the code appears in the output document!"><img src="https://jianweicheong.codeberg.page/media/posts/tools-academic/quarto.png" class="img-fluid figure-img" alt="The output of the code appears in the output document!"></a></p>
<figcaption>The output of the code appears in the output document!</figcaption>
</figure>
</div>
<p>This allows for reproducible research where the codes and outputs are contained in the same research write-up, and is perfect for cases where one wants to share technical reports complete with implementation details to a colleague.</p>
<p>But this is not all! Even without using the ability to run code in your documents, Quarto can still do <a href="https://quarto.org/docs/gallery/">a lot more</a>, e.g., writing books, creating dashboards for data science, or creating web apps to interact with data<sup>4</sup>. One can also publish to <a href="https://quartopub.com/">Quarto Pub</a> to share their Quarto documents easily.</p>
<div class="no-row-height column-margin column-container"><div id="fn4"><p><sup>4</sup>&nbsp;In fact, this very website is written in Quarto!</p></div></div></section>
<section id="typst" class="level2 page-columns page-full">
<h2 class="anchored" data-anchor-id="typst">Typst</h2>
<p><a href="https://typst.app/">Typst</a> is a modern typesetting system that aims to be an alternative to LaTeX. Fast, easy, sensible, with an actual scripting language, and actively developed, it has completely replaced LaTeX for me except for papers that are submitted for publication<sup>5</sup>. I even made a conference poster recently with Typst<sup>6</sup>.</p>
<div class="no-row-height column-margin column-container"><div id="fn5"><p><sup>5</sup>&nbsp;As far as I am aware, <a href="https://typst.app/blog/2025/typst-at-ijimai">IJIMAI is the only journal that accept Typst</a> at this point of time. Though, it seems that <a href="https://www.youtube.com/watch?v=zNZlAbCOjd8">arXiv has also taken an interest in Typst</a>.</p></div><div id="fn6"><p><sup>6</sup>&nbsp;The package <a href="https://typst.app/universe/package/peace-of-posters">peace-of-posters</a> can be used to easily create posters. <a href="../media/posts/tools-academic/poster_dde2025.svg" class="lightbox" data-gallery="quarto-lightbox-gallery-4"><img src="https://jianweicheong.codeberg.page/media/posts/tools-academic/poster_dde2025.svg" class="img-fluid"></a></p></div></div><p>Not only does Typst has a simple Markdown-like syntax, it is also extremely fast, allowing almost instant live previews while typing.</p>
<div class="quarto-figure quarto-figure-center">
<figure class="figure">
<p><video src="../media/posts/tools-academic/typst.webm" class="img-fluid" controls=""><a href="../media/posts/tools-academic/typst.webm">Video</a></video></p>
<figcaption>Live preview in Typst.</figcaption>
</figure>
</div>
<p>Typst also has an active <a href="https://typst.app/universe/">package ecosystem</a>, from drawing diagrams with <a href="https://typst.app/universe/package/cetz/">CeTZ</a>, creating presentation slides with <a href="https://typst.app/universe/package/touying/">Touying</a>, drawing quantum circuits with <a href="https://typst.app/universe/package/quill">Quill</a>, to plotting with <a href="https://typst.app/universe/package/lilaq">Lilaq</a>, Typst covers my entire workflow. My research notes that I share with my colleagues are written in Typst, I make diagrams in Typst, and I make worksheets for students in Typst.</p>
<p>Typst has also gained enough popularity that Pandoc and Quarto now support it as a PDF backend in place of LaTeX. It even comes preinstalled in Quarto. Unlike LaTeX, there are almost no wait time in compiling a document and error messages are not cryptic, making Typst a good PDF backend for Quarto<sup>7</sup>.</p>
<div class="no-row-height column-margin column-container"><div id="fn7"><p><sup>7</sup>&nbsp;Take a look at the talks <a href="https://www.youtube.com/watch?v=t30sqFch2v0">“Never again in outer par mode”</a> from Posit Conference 2023 and <a href="https://www.youtube.com/watch?v=2DbuqYKOsrY">“Styling Quarto PDFs with Typst”</a> from SatRdays London 2024, where they discusses the advantages of using Typst over LaTeX in Quarto</p></div></div></section>
<section id="an-rss-feed-reader" class="level2 page-columns page-full">
<h2 class="anchored" data-anchor-id="an-rss-feed-reader">An RSS feed reader</h2>
<p>You most likely have come across this little orange icon <img src="https://upload.wikimedia.org/wikipedia/en/thumb/4/43/Feed-icon.svg/20px-Feed-icon.svg.png" class="img-fluid"> in your time on the internet. In fact, there is one at the top right of this very site (though it’s not orange). This is <a href="https://en.wikipedia.org/wiki/RSS">RSS</a> (Really Simple Syndication), a standardized format to aggregate news or contents into a single feed.</p>
<p>Imagine having a single feed where you can read articles from multiple sources such as news websites, right when they are published. This is not an uncommon concept as some smartphones such as the iPhone provides such features with their news app. So how is this useful for academics?</p>
<p>Well, turns out journals typically provide a link to subscribe to their RSS feed too. One can then easily keep up with current research in journals relevant to their fields by aggregating them into a single RSS feed reader.</p>
<div class="quarto-figure quarto-figure-center">
<figure class="figure">
<p><a href="../media/posts/tools-academic/rss1.png" class="lightbox" data-gallery="quarto-lightbox-gallery-5" title="The RSS feed reader Newsraft showing new published papers in Physical Review Letters."><img src="https://jianweicheong.codeberg.page/media/posts/tools-academic/rss1.png" class="img-fluid figure-img" alt="The RSS feed reader Newsraft showing new published papers in Physical Review Letters."></a></p>
<figcaption>The RSS feed reader <a href="https://codeberg.org/newsraft/newsraft">Newsraft</a> showing new published papers in Physical Review Letters.</figcaption>
</figure>
</div>
<p>I check the feed every day, reading through the titles, and then the abstracts for titles that caught my attention. If I find the paper interesting or relevant, I can easily open the paper on a browser with a single hotkey to download or add it to my bibliography manager.</p>
<p>I am subscribed to 11 journals, totaling to around 50 to 100 new papers everyday (arXiv takes up the bulk of this number), a number that is small enough that it’s not a hassle to go through the feed every day<sup>8</sup>.</p>
<div class="no-row-height column-margin column-container"><div id="fn8"><p><sup>8</sup>&nbsp;I’m also subscribed to some news site and even some YouTube channels, eliminating the need for an account.</p></div></div><div class="quarto-figure quarto-figure-center">
<figure class="figure">
<p><a href="../media/posts/tools-academic/rss2.png" class="lightbox" data-gallery="quarto-lightbox-gallery-6" title="Reading the abstract of an arXiv paper in Newsraft."><img src="https://jianweicheong.codeberg.page/media/posts/tools-academic/rss2.png" class="img-fluid figure-img" alt="Reading the abstract of an arXiv paper in Newsraft."></a></p>
<figcaption>Reading the abstract of an arXiv paper in <a href="https://codeberg.org/newsraft/newsraft">Newsraft</a>.</figcaption>
</figure>
</div>
<p>Given how important keeping up with current developments in the field is for a researcher, an RSS feed reader (and there are many) makes the aggravating task of checking for new papers effortless, and greatly increases one’s exposure to current research.</p>
</section>
<section id="shiny" class="level2 page-columns page-full">
<h2 class="anchored" data-anchor-id="shiny">Shiny</h2>
<p><a href="https://shiny.posit.co/">Shiny</a> is an interactive and reactive dashboard/web app package that came out of the R community like Quarto (it is now also available for Python).</p>
<p>For most academics, in addition to exploring, analyzing, and visualizing your data and results by yourself, you will have to present and discuss your results with your colleagues. Most of the time such discussions take place with an opened PDF document or a couple of slides with plots. It is also not uncommon for someone to ask a question where the only answer is:</p>
<p>“I’ll go back and check.”,</p>
<p>even when you already have the data, because opening your favorite programming language and plotting it will take too long, leaving everyone in awkward silence while you embarrass yourself with your programming skills.</p>
<p>Here’s the solution: instead of static plots, use Shiny to present your results<sup>9</sup>!</p>
<div class="no-row-height column-margin column-container"><div id="fn9"><p><sup>9</sup>&nbsp;Or any other interactive dashboard app, e.g., there’s <a href="https://plotly.com/dash/">Dash</a> by Plotly, or <a href="https://panel.holoviz.org/">Panel</a> by HoloViz, or heck even Jupyter widgets.</p></div></div><div class="quarto-figure quarto-figure-center">
<figure class="figure">
<p><a href="../media/posts/tools-academic/shiny_demo.png" class="lightbox" data-gallery="quarto-lightbox-gallery-7" title="A Shiny web app with interactive and reactive elements I made for a class discussing the logistic map. See more examples here."><img src="https://jianweicheong.codeberg.page/media/posts/tools-academic/shiny_demo.png" class="img-fluid figure-img" alt="A Shiny web app with interactive and reactive elements I made for a class discussing the logistic map. See more examples here."></a></p>
<figcaption>A Shiny web app with interactive and reactive elements I made for a class discussing the <a href="https://en.wikipedia.org/wiki/Logistic_map">logistic map</a>. See more examples <a href="https://shiny.posit.co/r/gallery/">here</a>.</figcaption>
</figure>
</div>
<p>Shiny is quite powerful. You can even <a href="https://shiny.posit.co/r/gallery/widgets/file-upload/">let user upload say a CSV file</a> and let them <a href="https://shiny.posit.co/r/gallery/application-layout/plot-plus-three-columns/">choose which columns of the uploaded data to plot</a>. If you find the code too boilerplate-ish, you can also write <a href="https://quarto.org/docs/interactive/shiny/">Shiny in Quarto</a>, easily adding Markdown and LaTeX maths into your Shiny app.</p>
<p>As an interactive and reactive app, Shiny requires a server to run, i.e., typically your own laptop while presenting. But it is also possible to publish your Shiny app online at <a href="https://www.shinyapps.io/">shinyapps.io</a> or <a href="https://connect.posit.cloud/">Posit’s Connect Cloud</a><sup>10</sup>.</p>
<div class="no-row-height column-margin column-container"><div id="fn10"><p><sup>10</sup>&nbsp;Both of which are free with limitations, though <a href="https://docs.posit.co/shinyapps.io/guide/migration/">shinyapps.io will be migrating to Connect Cloud in 2027</a>.</p></div></div></section>
<section id="a-general-purpose-editor" class="level2 page-columns page-full">
<h2 class="anchored" data-anchor-id="a-general-purpose-editor">A general purpose editor</h2>
<p>Typically, one uses different <a href="https://en.wikipedia.org/wiki/Integrated_development_environment">integrated development environments</a> (IDEs) for different purposes, e.g., <a href="https://texstudio.org/">TeXstudio</a> or <a href="https://www.overleaf.com/">Overleaf</a> for LaTeX, <a href="https://posit.co/downloads">RStudio</a> for R, <a href="https://www.spyder-ide.org/">Spyder</a> or <a href="https://jupyter.org/">Jupyter</a> for Python, etc.</p>
<p>However, there are general purpose editors that can function as IDEs for almost anything. One popular example is <a href="https://code.visualstudio.com/">VS Code</a><sup>11</sup>, which tops the <a href="https://survey.stackoverflow.co/2025/technology#1-dev-id-es">Stack Overflow survey</a> as the most used editor almost every year.</p>
<div class="no-row-height column-margin column-container"><div id="fn11"><p><sup>11</sup>&nbsp;VS Code is what most people starts with, but use <a href="https://vscodium.com/">VSCodium</a> if you want VS Code without Microsoft’s telemetry and tracking.</p></div><div id="fn12"><p><sup>12</sup>&nbsp;As shown in the screenshot/video examples for Pandoc, Quarto, and Typst above</p></div></div><p>Other well-known editors that comes up in most online discourse are probably <a href="https://www.sublimetext.com/">Sublime Text</a>, <a href="https://en.wikipedia.org/wiki/Vim_(text_editor)">Vim</a>/<a href="https://neovim.io/">Neovim</a>, and <a href="https://www.gnu.org/software/emacs/">Emacs</a>. Personally, I am using <a href="https://helix-editor.com/">Helix</a><sup>12</sup> and Emacs.</p>
<p>Having a single general purpose editor makes entry to new programming languages or markup languages easy; why learn different tools for different programming languages when they are just plain text at the end of the day? There’s no need to download a new program, just start writing in what you are already comfortable with.</p>
<p>The editor will then be just another general purpose tool on your computer that you are already comfortable with, just like your file explorer, browser, image/video viewer, notepad etc., improving your productivity.</p>
<p>This is even more true for a Linux user, who might write simple shell scripts or edit config files frequently. This is partly why there are very enthusiastic Vim/Neovim and Emacs communities in the Linux space<sup>13</sup>.</p>
<div class="no-row-height column-margin column-container"><div id="fn13"><p><sup>13</sup>&nbsp;Here are some examples of people being enthusiastic about their editors. Vim/Neovim: <a href="https://www.youtube.com/watch?v=AAkrmfkC1L4">link</a>, <a href="https://www.youtube.com/watch?v=ST7vnfKjfvY">link</a>, <a href="https://www.youtube.com/watch?v=DOtM1mrWjUo">link</a>, <a href="https://www.youtube.com/watch?v=RmEtH5FQs28">link</a>. Emacs: <a href="https://www.youtube.com/watch?v=DMbrNhx2zWQ">link</a>, <a href="https://www.youtube.com/watch?v=urcL86UpqZc">link</a>.</p></div></div><p>Speaking of Linux…</p>
</section>
<section id="linux" class="level2 page-columns page-full">
<h2 class="anchored" data-anchor-id="linux">Linux</h2>
<p>Free<sup>14</sup> and open source, highly customizable, actually respects you instead of shoving products and ads down your throat, actually own and control your tools instead of the other way round, doesn’t spy on you, no bloatware with random preinstalled programs that you don’t care about.</p>
<div class="no-row-height column-margin column-container"><div id="fn14"><p><sup>14</sup>&nbsp;both in terms of <a href="https://en.wikipedia.org/wiki/Free_software">freedom</a> and price.</p></div></div><p>Honestly, there are so many good reasons to use Linux. But on a more practical level, Linux opens the doors to more efficient and productive workflows.</p>
<p>Perhaps you might find it hard to get started with all the tools that we talked about above. Going to each of their website, look for a download button, running the .exe to install it, figure out where and how you should be writing the Markdown or Typst codes, figure out how to compile it from a command prompt, only to see an error because the command prompt is not at the same directory as your file etc.</p>
<p>All these complications is one of the reason why most people use webapp over local installations, e.g., Overleaf vs.&nbsp;a local installation of LaTeX. More specifically, I believe the main roadblock seems to be the lack of a good general purpose editor and an accessible terminal in the typical workflow of Windows users.</p>
<p>Furthermore, installation of programs is extremely easy in most Linux <a href="https://en.wikipedia.org/wiki/Linux_distribution">distros</a>. For example, here’s how I install Firefox, Pandoc, Typst, LaTeX, R, and Julia all in one go in <a href="https://archlinux.org/">Arch Linux</a><sup>15</sup>:</p>
<div class="no-row-height column-margin column-container"><div id="fn15"><p><sup>15</sup>&nbsp;Want to use C/C++ or Python? Guess what, they are already preinstalled in almost all Linux distros.</p></div></div><div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb1" style="background: #f1f3f5;"><pre class="sourceCode bash code-with-copy"><code class="sourceCode bash"><span id="cb1-1"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">sudo</span> pacman <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">-S</span> firefox pandoc typst texlive r julia</span></code></pre></div></div>
<p>and to update all programs installed:</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb2" style="background: #f1f3f5;"><pre class="sourceCode bash code-with-copy"><code class="sourceCode bash"><span id="cb2-1"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">sudo</span> pacman <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">-Syu</span></span></code></pre></div></div>
<p>Ultimately, I find the accessibility and efficiency in installing and working with various programs an important factor in whether academics try, utilize, and tinker with tools that can improve their workflow and productivity, and Linux is the gateway to these<sup>16</sup>.</p>
<div class="no-row-height column-margin column-container"><div id="fn16"><p><sup>16</sup>&nbsp;You might be worried about Windows-exclusive software not working on Linux. These do exists, but I find the lost of Linux-exclusive software on a Windows machine worse, e.g., my PDF viewer <a href="https://github.com/pwmt/zathura">zathura</a>, bibliography manager <a href="https://codeberg.org/lukeflo/bibiman">bibiman</a>, terminal emulator <a href="https://codeberg.org/dnkl/foot">foot</a>, and window compositor <a href="https://github.com/niri-wm/niri">niri</a>.</p></div></div><p>You might not be convinced by this. But imagine having to do your work on a smart phone instead of a PC. The lost accessibility to software and the inefficiency in working with them will greatly affect how you work as compared to on a PC. The transition from Windows to Linux will feel the same for an experienced Linux user.</p>
</section>
<section id="marimo-over-jupyter" class="level2 page-columns page-full">
<h2 class="anchored" data-anchor-id="marimo-over-jupyter">Marimo over Jupyter</h2>
<p>Here’s a bonus one.</p>
<p>I take the somewhat popular yet controversial opinion that Jupyter notebooks are a scourge to scientific research. There are some annoying issues, such as how it’s saved as JSON instead of plain Python files, and how it teaches bad programming practices and habits to scientists. The latter can easily compound with carelessness to lead to invisible bugs and mistakes that produce wrong results, which might even end up in published research<sup>17</sup>!</p>
<div class="no-row-height column-margin column-container"><div id="fn17"><p><sup>17</sup>&nbsp;Not many researchers actually write <a href="https://en.wikipedia.org/wiki/Unit_testing">unit tests</a> for their spaghetti code function.</p></div></div><p>But much more troubling is the fact that Jupyter notebooks increase the chances of making these mistakes due to the creation of hidden states. In fact, a study showed that out of over 800000+ valid notebooks on GitHub, only ~24% are executed without errors, and only ~4% produced the same results <span class="citation" data-cites="PimentelJupyter2019">(Pimentel et al. 2019)</span>.</p>
<p>Perhaps this warrant a separate post<sup>18</sup>, but in short, if one wants to present some results with their corresponding codes, use Quarto, and if one really require a notebook environment, use <a href="https://marimo.io/">Marimo</a>, an alternative to Jupyter notebooks (only for Python).</p>
<div class="no-row-height column-margin column-container"><div id="fn18"><p><sup>18</sup>&nbsp;For now, take a look at <a href="https://www.youtube.com/watch?v=7jiPeIFXb6U">this talk</a> from JupyterCon 2018 to understand some of the problems with Jupyter notebooks, as well as this <a href="https://docs.marimo.io/faq/#what-problems-does-marimo-solve">FAQ from Marimo</a>.</p></div></div><p>The main difference between Marimo notebooks and Jupyter notebooks is the fact that the former is <em>reactive</em>. What this means is that Marimo notebooks always run in the correct order and always rerun the entire notebook on edits, eliminating the hidden states that plagues Jupyter notebooks.</p>
<p>If a code cell defining a variable is deleted, but the variable is used elsewhere, an error will be thrown. On the other hand, Jupyter notebooks continue to sneakily use that variable even when the programmer meant to remove it from the program.</p>
<p>A researcher should be aware of how certain shortcomings of their tools can be detrimental to their work, and should be open to finding alternatives or engage in self-improvement to eliminate them to minimize bad research outputs.</p>
<p>For example, learning to write properly unit tested functions instead of spaghetti codes, or learning to use type hinting in Python (which has been around since version 3.5) to pair with type checkers to minimize invisible bugs caused by carelessness that can lead to wrong published results.</p>



</section>


<a onclick="window.scrollTo(0, 0); return false;" id="quarto-back-to-top"><i class="bi bi-arrow-up"></i> Back to top</a><div id="quarto-appendix" class="default"><section class="quarto-appendix-contents" id="quarto-bibliography"><h2 class="anchored quarto-appendix-heading">References</h2><div id="refs" class="references csl-bib-body hanging-indent" data-entry-spacing="0">
<div id="ref-PimentelJupyter2019" class="csl-entry">
Pimentel, Joao Felipe, Leonardo Murta, Vanessa Braganholo, and Juliana Freire. 2019. <span>“A Large-Scale Study about Quality and Reproducibility of Jupyter Notebooks.”</span> In <em>2019 IEEE/ACM 16th International Conference on Mining Software Repositories (MSR)</em>. IEEE. <a href="https://doi.org/10.1109/msr.2019.00077">https://doi.org/10.1109/msr.2019.00077</a>.
</div>
</div></section></div> ]]></description>
  <category>software</category>
  <guid>https://jianweicheong.codeberg.page/posts/tools-academic.html</guid>
  <pubDate>Sat, 07 Jun 2025 16:00:00 GMT</pubDate>
</item>
<item>
  <title>An honest explanation of quantum computing</title>
  <link>https://jianweicheong.codeberg.page/posts/quantum-computing.html</link>
  <description><![CDATA[ 




<p>We have all heard it before:</p>
<div class="page-columns page-full"><blockquote class="blockquote">
<p>“It’s doing all possible computations simultaneously”<br>
“A quantum bit can be both 0 and 1 at the same time”<br>
“Quantum computers compute in parallel universes”<sup>1</sup></p>
</blockquote><div class="no-row-height column-margin column-container"><div id="fn1"><p><sup>1</sup>&nbsp;I’m shocked that this is <a href="https://blog.google/technology/research/google-willow-quantum-chip/">no longer an exaggeration</a>.</p></div></div></div>
<p>I get it, it’s difficult to talk about quantum mechanics without resorting to technical jargon, forcing us to rely on analogies that barely make sense.</p>
<p>But I mean, if it’s doing multiple computations simultaneously (in parallel universes or not), how many computations (or parallel universes) is it exactly?</p>
<p>And we can already do computations in “parallel universes” with conventional computers. It’s called multithreading, multiprocessing, or parallel computing.</p>
<p>So does this means that I can achieve the same advantage in principle if I simply match the number of conventional computers to the number of parallel universes? Does this means that quantum computers have <img src="https://latex.codecogs.com/png.latex?O(1)"> computational complexity?</p>
<p>You see, these popular explanations for how a quantum computer works are typically a gross simplification that fails when probed a little. The fact that they even exist in the first place surprises me because <strong>the concepts behind quantum computing is actually quite simple</strong>.</p>
<p>Yes, that’s right. I believe that the basics of quantum computing is simple enough that any reasonable layperson can understand it.</p>
<p>The mystique only arises when we conflate the metaphysical interpretations of quantum mechanics<sup>2</sup> with the explanations of the quantum computations that they can perform. This is akin to explaining classical computation with some metaphysical interpretation of the nature of electricity, e.g., “The reason your computer can calculate 1+1 is because electricity running in your computer can flow on top of each other and thus adds up.” which is not what happens at all.</p>
<div class="no-row-height column-margin column-container"><div id="fn2"><p><sup>2</sup>&nbsp;To be fair, these interpretations of quantum mechanics are indeed weird and mind-blowing. But again, no matter how weird electricity is, classical computer science is independent of it. Quantum computing should be viewed the same way. Also keep in mind that these are just interpretations with no evidence of being true.</p></div></div><p>So what exactly is this simple concept behind quantum computing?</p>
<p>It’s a sphere.</p>
<section id="enters-the-bloch-sphere" class="level1 page-columns page-full">
<h1>Enters the Bloch sphere</h1>
<div class="quarto-figure quarto-figure-center">
<figure class="figure">
<p><img src="https://jianweicheong.codeberg.page/media/posts/quantum-computing/bloch.svg" class="img-fluid figure-img"></p>
<figcaption>The Bloch sphere with an arrow inside pointing outwards from the center.</figcaption>
</figure>
</div>
<p>This is called a <a href="https://en.wikipedia.org/wiki/Bloch_sphere">Bloch sphere</a>, and it captures everything about a qubit (even all the alleged parallel universes).</p>
<p>Anyone can understand and visualize a sphere, so it puzzles me as to why we do not start with this instead of parallel universes or “being 0 and 1 at the same time”. The Bloch sphere is also mathematically accurate. That is, it’s not an analogy, it’s the truth!</p>
<p>In fact, the Bloch sphere also captures classical computing. The familiar logical 0 and 1 bits from classical computing are at the North and the South poles, respectively. In a conventional computer, imagine a bunch of these spheres with an arrow in each pointing only to either the North pole (bit 0) or the South pole (bit 1).</p>
<div class="quarto-figure quarto-figure-center">
<figure class="figure">
<p><img src="https://jianweicheong.codeberg.page/media/posts/quantum-computing/bits.svg" class="img-fluid figure-img"></p>
<figcaption>Eight Bloch spheres representing a sequence of eight classical bits.</figcaption>
</figure>
</div>
<p>During a classical computation, the arrows simply flip between the North and the South poles, most of the time conditioned on other arrows of other spheres<sup>3</sup>. The sequence of arrows then represents the results of the computation.</p>
<div class="no-row-height column-margin column-container"><div id="fn3"><p><sup>3</sup>&nbsp;E.g., a <a href="https://en.wikipedia.org/wiki/Inverter_(logic_gate)">NOT operation</a> flips 0 (North) to 1 (South) and 1 (South) to 0 (North), while the <a href="https://en.wikipedia.org/wiki/AND_gate">AND operation</a> flips a bit to 1 (South) only if the two input bits are both 1 (South).</p></div></div><p>What about a qubit?</p>
<p>Well, we simply free the arrow such that it is able to point to anywhere on the sphere. That’s it. Where the arrow points is the <em>state</em> of the qubit.</p>
<p>So when people say a qubit can be “0 and 1 at the same time”, what they actually mean is that the arrow is pointing somewhere between the North (0) and the South (1) poles. Typically this is at the equator, halfway between the poles.</p>
<p>Of course, there is not just one location on the equator of a sphere. Singapore is at the equator, so is Ecuador, and so are multiple other countries on Earth. This means that two qubits that are “0 and 1 at the same time” can be different. To differentiate them (figure out where their arrows point), or to pinpoint a location on Earth, we require two coordinates, the latitude, and the longitude.</p>
<p>Since the Bloch sphere is also a sphere, the “latitude” and the “longitude” are two variables (or degrees of freedom) that also defines a qubit: <img src="https://latex.codecogs.com/png.latex?%5Ctheta"> the “latitude”, and <img src="https://latex.codecogs.com/png.latex?%5Cvarphi"> the “longitude” also known as the relative phase.</p>
<div class="quarto-figure quarto-figure-center">
<figure class="figure">
<p><img src="https://jianweicheong.codeberg.page/media/posts/quantum-computing/qubits.svg" class="img-fluid figure-img"></p>
<figcaption>Eight Bloch spheres representing a sequence of eight qubits.</figcaption>
</figure>
</div>
<p>So what happens during a quantum computation?</p>
<p>Well, like the classical case, the arrow in each qubit simply jumps (or rotates) around to and from any point on the sphere<sup>4</sup>.</p>
<div class="no-row-height column-margin column-container"><div id="fn4"><p><sup>4</sup>&nbsp;In general, this is done by performing unitary operations (literally rotations) on it. Typically, these operations can be described by simpler <a href="https://en.wikipedia.org/wiki/Quantum_logic_gate">quantum logic gates</a> like the logic gates in the classical case.</p></div></div><p>So all these talks of a quantum computer being able to perform computations faster because of “superposition” or because it can performs computation “simultaneously” etc., simply refers to the fact that there is an increase in the degrees of freedom for the computation as compared to classical digital computations.</p>
<p>In other words, this freedom for the arrow (its state) to rotate and point to anywhere on the sphere, allows certain computing tasks to require less operations (flipping or rotating of arrows), than if the arrows are confined to the North and South poles.</p>
<p>But what’s with all the weirdness of quantum physics then?</p>
<p>It’s true that in actual quantum systems that implements a qubit, there are typically two preferred states (the North and South poles of our Bloch sphere). These are sometimes called <a href="https://en.wikipedia.org/wiki/Quantum_Darwinism"><em>pointer states</em></a>. We observes these preferred states in our classical world as they are what remains after the quantum system loses its <a href="https://en.wikipedia.org/wiki/Quantum_decoherence"><em>coherence</em></a>. But in reality, the quantum system sees no difference between these preferred states and any other states it could take (e.g., at the equator of our Bloch sphere).</p>
<p>Since we only observes these preferred states (and that our macroscopic reality is built out of them), we deemed them as “natural” which led to these apparent weirdness, e.g., how can something that takes the “natural” states of North and South, be placed in the “unintuitive” state of halfway between North and South?</p>
<p>So perhaps there are some credence to the weirdness and hype. However, again, just like how the computer science of classical computation is independent of the physics of electricity and semiconductors (or their interpretations) etc., the computer science of quantum computation should also be seen as separate from the interpretations of what’s happening in the physical systems that implements them.</p>
<div class="callout callout-style-default callout-note callout-titled">
<div class="callout-header d-flex align-content-center collapsed" data-bs-toggle="collapse" data-bs-target=".callout-1-contents" aria-controls="callout-1" aria-expanded="false" aria-label="Toggle callout">
<div class="callout-icon-container">
<i class="callout-icon"></i>
</div>
<div class="callout-title-container flex-fill">
<span class="screen-reader-only">Note</span>Extra notes
</div>
<div class="callout-btn-toggle d-inline-block border-0 py-1 ps-1 pe-0 float-end"><i class="callout-toggle"></i></div>
</div>
<div id="callout-1" class="callout-1-contents callout-collapse collapse">
<div class="callout-body-container callout-body">
<p>The attentive reader will notice that a sphere is spherically symmetric. Just like on the Earth, there are no real ups and downs except for those defined by convention.</p>
<p>What this means is that we can orient the Bloch sphere however we want. Pick any two points at opposite ends of the sphere and we can define that as the 0 and 1. The reason the North and the South poles are defined as the 0 and 1 is simply convention.</p>
<p>Now that you know this, you should realize that the idea that a qubit being “both 0 and 1 at the same time” somehow grants an advantage is quite silly, since it’s just another point on the sphere, no different from the North (0) and the South (1) poles.</p>
<p>A more attentive reader will also ask “what about the insides of the sphere?”.</p>
<p>The reader is right. There is actually a third degree of freedom: the “radius”, which describes the length of the arrow. So yes, the arrow can point to somewhere inside the sphere. These are called <em>mixed states</em> which are ensemble averages, whereas arrows that point to the surface of the sphere are called <em>pure states</em>.</p>
<p>During a quantum computation, the qubit is sensitive to thermal noises that rotate the arrow in a manner that is undesired. These noises cause the average behavior of the arrow to point to some fixed point inside the sphere depending on the temperature, in a process known as <a href="https://en.wikipedia.org/wiki/Quantum_decoherence"><em>decoherence</em></a>. This is why (depending on the implementation) quantum computers need to be kept at a very cold temperature, so that they can perform computations before they eventually reach the fixed point inside the sphere.</p>
</div>
</div>
</div>
</section>
<section id="entanglement-the-actual-spice" class="level1 page-columns page-full">
<h1>Entanglement — the actual spice</h1>
<p>The Bloch sphere might reminds one of analog computers, where quantities can take a continuum of values instead of the 0s and 1s in digital computers. In fact, if we can build an analog computer with two degrees of freedom like the Bloch sphere, then it can function exactly like a single qubit.</p>
<p>The “quantumness” only comes in if we include the concept of <em>entanglement</em><sup>5</sup>. Entanglement is the main spice that makes quantum systems special, preventing it from being replicated by classical systems, e.g., the analog computer with two degrees of freedom<sup>6</sup>.</p>
<div class="no-row-height column-margin column-container"><div id="fn5"><p><sup>5</sup>&nbsp;Entanglement is <a href="../posts/spooky-action.html">often conflated with the phrase “<em>spooky action at a distance</em>”</a>. The latter describes the apparent nonlocal collapse of the wave function. <span class="citation" data-cites="EinsteinEPR1935">Einstein, Podolsky, and Rosen (1935)</span> used entanglement as an example to illustrate the significance of this nonlocal collapse, which eventually led to this conflation.</p></div><div id="fn6"><p><sup>6</sup>&nbsp;This statement is slightly imprecise. Two classical systems indeed cannot be correlated in a way like entanglement can, but we can certainly simulate entanglement in classical systems.</p></div></div><p>Imagine what would happen if we have two qubits.</p>
<div class="quarto-figure quarto-figure-center">
<figure class="figure">
<p><img src="https://jianweicheong.codeberg.page/media/posts/quantum-computing/bloch2.svg" class="img-fluid figure-img"></p>
<figcaption>Two Bloch spheres.</figcaption>
</figure>
</div>
<p>One might think that there are now six variables to describe the pair of qubits (where both arrows point) since we now have two Bloch spheres<sup>7</sup>. However, this is only true if there is no correlation between the pair of qubits, that is, the two arrows are independent of each other.</p>
<div class="no-row-height column-margin column-container"><div id="fn7"><p><sup>7</sup>&nbsp;If you thought the number is four instead, take a look at the <strong>Extra notes</strong> above.</p></div></div><p>Typically, when the two qubits interact with each other (or undergo quantum computations together), they can become entangled, which means that their two arrows (their states) are now correlated in a way that is impossible to achieve classically.</p>
<p>This is akin to them becoming a single (hyper)sphere with a single arrow pointing at some point that represents their combined state. The dimension of this (hyper)sphere will be the number of variables or degrees of freedom required to describe them.</p>
<p>Quite surprisingly, there can be a total of up to 15 degrees of freedom!</p>
<div class="quarto-figure quarto-figure-center">
<figure class="figure">
<p><img src="https://jianweicheong.codeberg.page/media/posts/quantum-computing/bloch15.svg" class="img-fluid figure-img"></p>
<figcaption>It’s impossible to draw a 15 dimensional sphere, so here’s a fun visual instead.</figcaption>
</figure>
</div>
<p>Six variables describe the two qubits independently (three for each), while the remaining nine describes how they are correlated to each other<sup>8</sup>.</p>
<div class="no-row-height column-margin column-container"><div id="fn8"><p><sup>8</sup>&nbsp;For the learned reader, these are the 15 real coefficients that describe an arbitrary two qubit density matrix when written in the Pauli basis. The nines are the possible combinations of the three Pauli matrices.</p></div></div><p>It is important to note that this single 15 dimensional Bloch (hyper)sphere describes both qubits together, and it cannot be broken down into separate descriptions for each qubit (without losing information about the entanglement). This means that the two entangled qubits can only be described as single object.</p>
<p>This is the stark feature of quantum systems that sets it apart from classical systems. Supposed that we were to build two analog computers with three degrees of freedom each to simulate the two Bloch spheres, before attempting to correlate their states (perhaps by performing some joint operations on them). We can never correlate them to the extent that entanglement could<sup>9</sup>.</p>
<div class="no-row-height column-margin column-container"><div id="fn9"><p><sup>9</sup>&nbsp;For classical correlations (no entanglement), they can be represented as a weighted sum of multiple pairs of independent Bloch spheres. These are called <a href="https://en.wikipedia.org/wiki/Separable_state"><em>separable states</em></a>. Note however that we can of course still simulate or find a classical analog system with 15 degrees of freedom to simulate the two entangled qubits.</p></div></div><p>In addition to the increased degrees of freedom, this increased correlation gained by entanglement can also be exploited in various quantum computational tasks.</p>
</section>
<section id="is-that-it" class="level1 page-columns page-full">
<h1>Is that it?</h1>
<p>Yep, that’s it.</p>
<p>Quantum computation is simply the rotations of these arrows on/in Bloch spheres<sup>10</sup>. While classical computing is confined to flipping between the North and South poles, quantum computing allows arrows to freely rotate anywhere on the Bloch sphere.</p>
<div class="no-row-height column-margin column-container"><div id="fn10"><p><sup>10</sup>&nbsp;With some caveats like the fact that we can’t determine where the arrows point (read-out of the state) directly, which we did not touch upon.</p></div></div><p>Where the arrows point and how they rotate is how we store and process information, just like how a sequence of North and South poles (0s and 1s) and how they flip, encodes, and processes information in the classical case.</p>
<p>By combining multiple qubits, the allowable space or degrees of freedom for this information processing increases. Combined with entanglement, this information processing can take place in even higher dimensional space, with some algorithms exploiting the increased correlation.</p>
<p>The main “essence” of quantum computing is this change in computing paradigm, regardless of whether quantum systems that implement it do so with the help of parallel universes or not.</p>
<p>Again, I want to emphasize the separation between the implementation’s physics (and its interpretation), and this new computing paradigm that grants these advantages over classical computing.</p>
<p>The difficulty of course, is coming up with algorithms that not only can utilize the increased degrees of freedom, but also can be better or faster than the classical case. It turns out however, that classical computing (being confined to the North and South poles) is good enough in most cases, and quantum algorithms that can be better are rare.</p>
<p>This might be because in classical digital computing, how we encode and process information are quite straightforward, and decades of creating higher abstractions means that no programmers are flipping individual bits. However, in the relatively young field of quantum computation, physicists are still concerned with the individual operations or rotations to perform on individual or groups of qubits.</p>
<p>Perhaps one day quantum computing (or even computing with larger degrees of freedom) will reach the same status as classical computing. But for now, we are stuck with rotating arrows.</p>



</section>


<a onclick="window.scrollTo(0, 0); return false;" id="quarto-back-to-top"><i class="bi bi-arrow-up"></i> Back to top</a><div id="quarto-appendix" class="default"><section class="quarto-appendix-contents" id="quarto-bibliography"><h2 class="anchored quarto-appendix-heading">References</h2><div id="refs" class="references csl-bib-body hanging-indent" data-entry-spacing="0">
<div id="ref-EinsteinEPR1935" class="csl-entry">
Einstein, A., B. Podolsky, and N. Rosen. 1935. <span>“Can Quantum-Mechanical Description of Physical Reality Be Considered Complete?”</span> <em>Physical Review</em> 47 (10): 777–80. <a href="https://doi.org/10.1103/physrev.47.777">https://doi.org/10.1103/physrev.47.777</a>.
</div>
</div></section></div> ]]></description>
  <category>physics</category>
  <guid>https://jianweicheong.codeberg.page/posts/quantum-computing.html</guid>
  <pubDate>Fri, 10 Jan 2025 16:00:00 GMT</pubDate>
</item>
<item>
  <title>Life as an entropic conspiracy</title>
  <link>https://jianweicheong.codeberg.page/posts/life-conspiracy.html</link>
  <description><![CDATA[ 




<section id="the-right-question-to-ask" class="level1 page-columns page-full">
<h1>The right question to ask</h1>
<blockquote class="blockquote">
<p>“What is the meaning of life?”</p>
</blockquote>
<p>This question has become quite a cliché, with an equally cliché answer of:</p>
<blockquote class="blockquote">
<p>“It’s whatever you make it out to be!”</p>
</blockquote>
<p>that has become the norm.</p>
<p>I believe the reason why such a cop-out answer is the norm is because deep down, we all understand that life has no intrinsic objective meaning<sup>1</sup>. In fact, given that the prerequisite for the concept of “meaning” to exist in the first place is the existence of life itself, this question seems rather like a trick question.</p>
<div class="no-row-height column-margin column-container"><div id="fn1"><p><sup>1</sup>&nbsp;Sans whatever religious/spiritual beliefs that one might have.</p></div><div id="fn2"><p><sup>2</sup>&nbsp;e.g., the aforementioned religious/spiritual beliefs.</p></div></div><p>Similar to how a system cannot demonstrate its own consistency in Gödel’s second incompleteness theorem, perhaps one must require a framework that is external to life itself to apply “meaning” for life’s existence, before life’s existence<sup>2</sup>.</p>
<p>Nevertheless, by replacing the quite human term of “meaning” with a more mechanistic term like “purpose”, we might be able to answer more reasonably the resulting question of:</p>
<div class="page-columns page-full"><blockquote class="blockquote">
<p>“What is the <strong>purpose</strong> of life?”<sup>3</sup></p>
</blockquote><div class="no-row-height column-margin column-container"><div id="fn3"><p><sup>3</sup>&nbsp;The term “purpose” might still be quite “agentic”, in the sense that it might mislead one into thinking that life has some agency over its own evolution based on some sense of purpose that it has. I assure that this is not what this post is about.</p></div></div></div>
</section>
<section id="a-physicists-perspective-the-conspiracy" class="level1 page-columns page-full">
<h1>A Physicist’s perspective — the conspiracy</h1>
<p>The study of life is typically assumed to fall under the domain of Biology. However, physicists have also been attempting to uncover a physical principle that leads to life’s emergence and its behaviors <span class="citation" data-cites="PrigogineBiological1971">(Prigogine and Nicolis 1971)</span>. A hint lies in the stark difference between life and inanimate matter: life’s ability to efficiently absorb or consume useful energy and dissipating it as heat, resulting in increased entropy production.</p>
<p>While this have been pointed out by Erwin Schrödinger in his seminar book <a href="https://en.wikipedia.org/wiki/What_Is_Life%3F"><em>What is life?</em></a> back in the 1940s, it has undergone rigorous thermodynamic treatment in recent years, thanks to advances in the physics of non-equilibrium thermodynamics <span class="citation" data-cites="CollinVerification2005 MichaelianThermodynamic2011">(Collin et al. 2005; K. Michaelian 2011)</span>.</p>
<p>In short, in the presence of an external non-equilibrium energy source (like the Sun), systems adapt and self-organize themselves to better absorb energy and dissipate heat or entropy. This process has come to be known as <em>dissipation-driven adaptation</em> <span class="citation" data-cites="EnglandStatistical2013 PerunovStatistical2016">(England 2013; Perunov, Marsland, and England 2016)</span>.</p>
<p>Life is hypothesized to be such a dissipative system that serves to increase entropy production.</p>
<p>From the birth of planetoids in the dusts of protoplanetary disks to the formation of vortices we see in chaotic turbulent flows such as hurricanes, stable structures with low internal entropy arise out of such chaotic interactions in order to more efficiently dissipate entropy.</p>
<p>Life might be one of such order out of chaos, a long-running, universe-scale conspiracy to increase the entropy of the cosmos.</p>
<section id="self-organization-order-out-of-chaos" class="level2 page-columns page-full">
<h2 class="anchored" data-anchor-id="self-organization-order-out-of-chaos">Self-organization — order out of chaos</h2>
<p>Self-organization of complex structures in the presence of an external energy source have been demonstrated both experimentally and computationally. In a computational simulation involving 20 particles in a simplified chemical space, complex bonding patterns emerged under resonance with a periodic drive, increasing the rate of work absorption and heat dissipation <span class="citation" data-cites="KachmanSelfOrganized2017">(Kachman, Owen, and England 2017)</span>.</p>

<div class="no-row-height column-margin column-container"><div class="">
<div class="quarto-figure quarto-figure-center">
<figure class="figure">
<p><a href="../media/posts/life-conspiracy/kondepudi.png" class="lightbox" data-gallery="quarto-lightbox-gallery-1" title="Reprinted from @KondepudiEnddirected2015"><img src="https://jianweicheong.codeberg.page/media/posts/life-conspiracy/kondepudi.png" class="img-fluid figure-img" alt="Reprinted from Kondepudi, Kay, and Dixon (2015)"></a></p>
<figcaption>Reprinted from <span class="citation" data-cites="KondepudiEnddirected2015">Kondepudi, Kay, and Dixon (2015)</span></figcaption>
</figure>
</div>
<p>Setup and result of self-organizing conducting beads.</p>
</div></div><p>Experimentally, conducting beads 4mm in diameter immersed in oil in a petri dish were shown to self-organize to a complex, tree-like structure when subjected to a high voltage. This occurred at a critical point and was accompanied by a spike in entropy dissipation <span class="citation" data-cites="KondepudiEnddirected2015">(Kondepudi, Kay, and Dixon 2015)</span>. Silver nanorods were also observed to self-organized in the presence of a doughnut-shaped laser source, with resulting structures depending on the wavelengths and polarization of the laser source <span class="citation" data-cites="ItoSelective2013">(Ito et al. 2013)</span>.</p>
<p>Likewise, by considering UVC light of 230nm to 270nm from the Sun as the non-equilibrium energy source (a range of wavelengths that were most intense in the early Earth), it was demonstrated that RNA polymers could proliferate via photochemical reactions, and that DNA polymers would denature from absorption and dissipation at these UVC wavelengths. These are crucial steps for self-replication without enzymes <span class="citation" data-cites="MichaelianMicroscopic2017 MichaelianUVC2019">(Karo Michaelian 2017; Karo Michaelian and Padilla 2019)</span>.</p>
<p>Homochirality in DNA and RNA was also shown to be achieved via photochemical reactions with UVC wavelengths light <span class="citation" data-cites="MichaelianHomochirality2018">(Karo Michaelian 2018)</span>. Correlations also exist between the maximum absorbance wavelengths of early life’s nucleobases and amino acids, and UVC wavelengths <span class="citation" data-cites="MichaelianFundamental2015">(K. Michaelian and Simeonov 2015)</span>.</p>
<p>These findings suggest that in the RNA world during early Earth era, RNA or pre-RNA polymers may have self-organized or adapted to efficeintly absorb and dissipate UVC solar spectrum, a prime example of dissipative adaptation in the emergence of life.</p>
</section>
<section id="self-replication-the-more-the-merrier" class="level2">
<h2 class="anchored" data-anchor-id="self-replication-the-more-the-merrier">Self-replication — the more the merrier</h2>
<p>Clearly, the more dissipators there are in a system, the greater the overall entropic dissipation. It’s therefore unsurprising that complex systems can self-replicate to increase entropy dissipation.</p>
<p>In fact, even the act of self-replication itself can serves to dissipate heat, e.g., it was shown that <em>E. coli</em> bacteria dissipate at least five times as much heat during replication compared to individual heat dissipation <span class="citation" data-cites="EnglandStatistical2013">(England 2013)</span>.</p>
<p>The process of DNA replication (copolymerisation) also dissipates entropy depending on the information entropy of the DNA polymer. Information creation decreases the Shannon entropy of the polymer and generates mutual information with the original. In the process, entropy is dissipated to the environment as quantified by the affinity per copied nucleotide <span class="citation" data-cites="AndrieuxNonequilibrium2008">(Andrieux and Gaspard 2008)</span>.</p>
<p>In general, various dissipative systems exhibit self-replication behaviors. For example, computer simulations of colloidal clusters show that the exponential growth rates of self-replication varies with the energy landscape of the system <span class="citation" data-cites="ZeravcicSelfreplicating2014">(Zeravcic and Brenner 2014)</span>.</p>
<p>Three-dimensional vortices also self-replicates in rotating shear flows, as demonstrated in a numerical study <span class="citation" data-cites="MarcusThreeDimensional2013">(Marcus et al. 2013)</span>. This was also proposed to be how planetoids form in protoplanetary disks <span class="citation" data-cites="BarrancoThreeDimensional2005">(Barranco and Marcus 2005)</span>, whereby the larger system can replicate smaller dissipative systems in it, or eventually split into a copy or more of itself, i.e., disk fragmentation that forms binary star systems <span class="citation" data-cites="OffnerFormation2010">(Offner et al. 2010)</span>.</p>
<p>Given these observations, one might hypothesize that early organic compounds evolved the ability to self-replicate as a means to ensure the constant presence of entropy dissipating structures, driving the emergence and evolution of life.</p>
</section>
<section id="adaptation-resonance-with-the-environment" class="level2 page-columns page-full">
<h2 class="anchored" data-anchor-id="adaptation-resonance-with-the-environment">Adaptation — resonance with the environment</h2>
<p>If life indeed self-organizes to more efficiently absorb external energy sources and dissipate heat, we would expect it to adapt or change its structures in response to changes in those energy sources over time.</p>

<div class="no-row-height column-margin column-container"><div class="">
<div class="quarto-figure quarto-figure-center">
<figure class="figure">
<p><a href="../media/posts/life-conspiracy/michaelian.png" class="lightbox" data-gallery="quarto-lightbox-gallery-2" title="Reprinted from @MichaelianFundamental2015"><img src="https://jianweicheong.codeberg.page/media/posts/life-conspiracy/michaelian.png" class="img-fluid figure-img" alt="Reprinted from K. Michaelian and Simeonov (2015)"></a></p>
<figcaption>Reprinted from <span class="citation" data-cites="MichaelianFundamental2015">K. Michaelian and Simeonov (2015)</span></figcaption>
</figure>
</div>
<p>Correlations between solar spectrum and the emergence of organic compounds and cells in the five different periods. As Earth’s conditions changes, more complex organic compounds evolved to suit these conditions. The general trends are a shift from UVC absorbing to visible spectrum absorbing compounds, and a shift to oxygen based respiration.</p>
</div></div><p>This seems to be evident throughout life’s history, where its evolution has underwent avalanche dynamics with quick and sudden changes in complexity rather than smooth and gradual evolution <span class="citation" data-cites="PaczuskiAvalanche1996">(Paczuski, Maslov, and Bak 1996)</span>. Some even modeled these sudden transitions of evolution as a Big Bang model with “evolutionary inflation” akin to the cosmological inflation model <span class="citation" data-cites="KooninBiological2007">(Koonin 2007)</span>. Simulated artificial life have also demonstrated how the fitness or how the population adapts to the environment, can undergo multiple discontinuous jumps <span class="citation" data-cites="AdamiSelforganized1995">(Adami 1995)</span>.</p>
<p>Correlating Earth’s historical environmental changes (like solar spectrum, temperature, and atmospheric compositions) with organic compounds’ emergence and evolution shows that life has consistently adapted to better survive and absorb energy according to the prevailing condition in that period <span class="citation" data-cites="MichaelianFundamental2015">(K. Michaelian and Simeonov 2015)</span>.</p>
<p>An interesting example is the case of adenosine triphosphate (ATP) synthase, an enzyme that lies along the cell membrane of a cell that breaks down ATP to adenosine diphosphate (ADP) and a phosphate. This breakage releases energy that powers the cell.</p>
<p>Using a kinetic model of the ATP synthase process reveals that both its entropy production and the Shannon entropy of the transitional states, are maximized at a catalytic dwell angle<sup>4</sup> consistent with the actual ATP synthase’s angle, indicating that the ATP synthase evolved to optimize entropy production <span class="citation" data-cites="DewarFunctional2006">(Dewar, Juretić, and Županović 2006)</span>.</p>
<div class="no-row-height column-margin column-container"><div id="fn4"><p><sup>4</sup>&nbsp;The ATP synthase rotates in angles of <img src="https://latex.codecogs.com/png.latex?120%5E%7B%5Ccirc%7D"> for each ATP to ADP + P<img src="https://latex.codecogs.com/png.latex?_i"> reaction. However, at the moment of breakage of ATP bonds to ADP + P<img src="https://latex.codecogs.com/png.latex?_i">, the synthase will pause for a short amount of time of around 2ms <span class="citation" data-cites="UenoATP2005">(Ueno et al. 2005)</span>. The angle at which this pause happens is known as the catalytic dwell.</p></div><div id="fn5"><p><sup>5</sup>&nbsp;One should keep in mind that none of these results were shown for complex life such as us humans, and one should not have the illusion that this hints at some sort of “<a href="https://en.wikipedia.org/wiki/Lamarckism">Lamarckism</a>”.</p></div></div><p>These findings suggest that the process of self-organization to an external energy source is an ongoing, dynamic phenomenon that continues in response to changing conditions.<sup>5</sup></p>
</section>
</section>
<section id="the-purpose-of-life" class="level1">
<h1>The purpose of life</h1>
<p>So why does the universe favors orders such as life, rather than simply maintaining the disorder that we might expect?</p>
<p>As discussed, it turns out that these eddies of order and low entropy pockets of the universe such as life, are but a small price for the universe to pay to optimize the production of entropy in the long run.</p>
<p>In other words, contrary to what one might expect, the presence of stable complex structures such as life produces more entropy in the system than if these structures were absent.</p>
<p>So it seems that the conspiracy of life is an entropic one.</p>
<p>This long-running conspiracy is but an attempt to speed up and maximize the production of entropy and disorder. The universe has set aside a small part of itself for us, masquerading as an exception to the Second Law of thermodynamics, but is in fact a testament of its power.</p>
<p>This is the purpose of life, the unwitting servitude as an agent of entropy.</p>



</section>


<a onclick="window.scrollTo(0, 0); return false;" id="quarto-back-to-top"><i class="bi bi-arrow-up"></i> Back to top</a><div id="quarto-appendix" class="default"><section class="quarto-appendix-contents" id="quarto-bibliography"><h2 class="anchored quarto-appendix-heading">References</h2><div id="refs" class="references csl-bib-body hanging-indent" data-entry-spacing="0">
<div id="ref-AdamiSelforganized1995" class="csl-entry">
Adami, C. 1995. <span>“Self-Organized Criticality in Living Systems.”</span> <em>Physics Letters A</em> 203 (1): 29–32. <a href="https://doi.org/10.1016/0375-9601(95)00372-a">https://doi.org/10.1016/0375-9601(95)00372-a</a>.
</div>
<div id="ref-AndrieuxNonequilibrium2008" class="csl-entry">
Andrieux, David, and Pierre Gaspard. 2008. <span>“Nonequilibrium Generation of Information in Copolymerization Processes.”</span> <em>Proceedings of the National Academy of Sciences</em> 105 (28): 9516–21. <a href="https://doi.org/10.1073/pnas.0802049105">https://doi.org/10.1073/pnas.0802049105</a>.
</div>
<div id="ref-BarrancoThreeDimensional2005" class="csl-entry">
Barranco, Joseph A., and Philip S. Marcus. 2005. <span>“Three‐dimensional Vortices in Stratified Protoplanetary Disks.”</span> <em>The Astrophysical Journal</em> 623 (2): 1157–70. <a href="https://doi.org/10.1086/428639">https://doi.org/10.1086/428639</a>.
</div>
<div id="ref-CollinVerification2005" class="csl-entry">
Collin, D., F. Ritort, C. Jarzynski, S. B. Smith, I. Tinoco, and C. Bustamante. 2005. <span>“Verification of the Crooks Fluctuation Theorem and Recovery of RNA Folding Free Energies.”</span> <em>Nature</em> 437 (7056): 231–34. <a href="https://doi.org/10.1038/nature04061">https://doi.org/10.1038/nature04061</a>.
</div>
<div id="ref-DewarFunctional2006" class="csl-entry">
Dewar, R. C., D. Juretić, and P. Županović. 2006. <span>“The Functional Design of the Rotary Enzyme ATP Synthase Is Consistent with Maximum Entropy Production.”</span> <em>Chemical Physics Letters</em> 430 (1–3): 177–82. <a href="https://doi.org/10.1016/j.cplett.2006.08.095">https://doi.org/10.1016/j.cplett.2006.08.095</a>.
</div>
<div id="ref-EnglandStatistical2013" class="csl-entry">
England, Jeremy L. 2013. <span>“Statistical Physics of Self-Replication.”</span> <em>The Journal of Chemical Physics</em> 139 (12). <a href="https://doi.org/10.1063/1.4818538">https://doi.org/10.1063/1.4818538</a>.
</div>
<div id="ref-ItoSelective2013" class="csl-entry">
Ito, Syoji, Hiroaki Yamauchi, Mamoru Tamura, Shimpei Hidaka, Hironori Hattori, Taichi Hamada, Keisuke Nishida, et al. 2013. <span>“Selective Optical Assembly of Highly Uniform Nanoparticles by Doughnut-Shaped Beams.”</span> <em>Scientific Reports</em> 3 (1). <a href="https://doi.org/10.1038/srep03047">https://doi.org/10.1038/srep03047</a>.
</div>
<div id="ref-KachmanSelfOrganized2017" class="csl-entry">
Kachman, Tal, Jeremy A. Owen, and Jeremy L. England. 2017. <span>“Self-Organized Resonance During Search of a Diverse Chemical Space.”</span> <em>Physical Review Letters</em> 119 (3). <a href="https://doi.org/10.1103/physrevlett.119.038001">https://doi.org/10.1103/physrevlett.119.038001</a>.
</div>
<div id="ref-KondepudiEnddirected2015" class="csl-entry">
Kondepudi, Dilip, Bruce Kay, and James Dixon. 2015. <span>“End-Directed Evolution and the Emergence of Energy-Seeking Behavior in a Complex System.”</span> <em>Physical Review E</em> 91 (5). <a href="https://doi.org/10.1103/physreve.91.050902">https://doi.org/10.1103/physreve.91.050902</a>.
</div>
<div id="ref-KooninBiological2007" class="csl-entry">
Koonin, Eugene V. 2007. <span>“The Biological Big Bang Model for the Major Transitions in Evolution.”</span> <em>Biology Direct</em> 2 (1): 21. <a href="https://doi.org/10.1186/1745-6150-2-21">https://doi.org/10.1186/1745-6150-2-21</a>.
</div>
<div id="ref-MarcusThreeDimensional2013" class="csl-entry">
Marcus, Philip S., Suyang Pei, Chung-Hsiang Jiang, and Pedram Hassanzadeh. 2013. <span>“Three-Dimensional Vortices Generated by Self-Replication in Stably Stratified Rotating Shear Flows.”</span> <em>Physical Review Letters</em> 111 (8). <a href="https://doi.org/10.1103/physrevlett.111.084501">https://doi.org/10.1103/physrevlett.111.084501</a>.
</div>
<div id="ref-MichaelianThermodynamic2011" class="csl-entry">
Michaelian, K. 2011. <span>“Thermodynamic Dissipation Theory for the Origin of Life.”</span> <em>Earth System Dynamics</em> 2 (1): 37–51. <a href="https://doi.org/10.5194/esd-2-37-2011">https://doi.org/10.5194/esd-2-37-2011</a>.
</div>
<div id="ref-MichaelianMicroscopic2017" class="csl-entry">
Michaelian, Karo. 2017. <span>“Microscopic Dissipative Structuring and Proliferation at the Origin of Life.”</span> <em>Heliyon</em> 3 (10): e00424. <a href="https://doi.org/10.1016/j.heliyon.2017.e00424">https://doi.org/10.1016/j.heliyon.2017.e00424</a>.
</div>
<div id="ref-MichaelianHomochirality2018" class="csl-entry">
———. 2018. <span>“Homochirality Through Photon-Induced Denaturing of RNA/DNA at the Origin of Life.”</span> <em>Life</em> 8 (2): 21. <a href="https://doi.org/10.3390/life8020021">https://doi.org/10.3390/life8020021</a>.
</div>
<div id="ref-MichaelianUVC2019" class="csl-entry">
Michaelian, Karo, and Norberto Santillán Padilla. 2019. <span>“UVC Photon-Induced Denaturing of DNA: A Possible Dissipative Route to Archean Enzyme-Less Replication.”</span> <em>Heliyon</em> 5 (6): e01902. <a href="https://doi.org/10.1016/j.heliyon.2019.e01902">https://doi.org/10.1016/j.heliyon.2019.e01902</a>.
</div>
<div id="ref-MichaelianFundamental2015" class="csl-entry">
Michaelian, K., and A. Simeonov. 2015. <span>“Fundamental Molecules of Life Are Pigments Which Arose and Co-Evolved as a Response to the Thermodynamic Imperative of Dissipating the Prevailing Solar Spectrum.”</span> <em>Biogeosciences</em> 12 (16): 4913–37. <a href="https://doi.org/10.5194/bg-12-4913-2015">https://doi.org/10.5194/bg-12-4913-2015</a>.
</div>
<div id="ref-OffnerFormation2010" class="csl-entry">
Offner, Stella S. R., Kaitlin M. Kratter, Christopher D. Matzner, Mark R. Krumholz, and Richard I. Klein. 2010. <span>“THE FORMATION OF LOW-MASS BINARY STAR SYSTEMS VIA TURBULENT FRAGMENTATION.”</span> <em>The Astrophysical Journal</em> 725 (2): 1485–94. <a href="https://doi.org/10.1088/0004-637x/725/2/1485">https://doi.org/10.1088/0004-637x/725/2/1485</a>.
</div>
<div id="ref-PaczuskiAvalanche1996" class="csl-entry">
Paczuski, Maya, Sergei Maslov, and Per Bak. 1996. <span>“Avalanche Dynamics in Evolution, Growth, and Depinning Models.”</span> <em>Physical Review E</em> 53 (1): 414–43. <a href="https://doi.org/10.1103/physreve.53.414">https://doi.org/10.1103/physreve.53.414</a>.
</div>
<div id="ref-PerunovStatistical2016" class="csl-entry">
Perunov, Nikolay, Robert A. Marsland, and Jeremy L. England. 2016. <span>“Statistical Physics of Adaptation.”</span> <em>Physical Review X</em> 6 (2). <a href="https://doi.org/10.1103/physrevx.6.021036">https://doi.org/10.1103/physrevx.6.021036</a>.
</div>
<div id="ref-PrigogineBiological1971" class="csl-entry">
Prigogine, I., and G. Nicolis. 1971. <span>“Biological Order, Structure and Instabilities.”</span> <em>Quarterly Reviews of Biophysics</em> 4 (2–3): 107–48. <a href="https://doi.org/10.1017/s0033583500000615">https://doi.org/10.1017/s0033583500000615</a>.
</div>
<div id="ref-UenoATP2005" class="csl-entry">
Ueno, Hiroshi, Toshiharu Suzuki, Kazuhiko Kinosita, and Masasuke Yoshida. 2005. <span>“ATP-Driven Stepwise Rotation of f o f 1 -ATP Synthase.”</span> <em>Proceedings of the National Academy of Sciences</em> 102 (5): 1333–38. <a href="https://doi.org/10.1073/pnas.0407857102">https://doi.org/10.1073/pnas.0407857102</a>.
</div>
<div id="ref-ZeravcicSelfreplicating2014" class="csl-entry">
Zeravcic, Zorana, and Michael P. Brenner. 2014. <span>“Self-Replicating Colloidal Clusters.”</span> <em>Proceedings of the National Academy of Sciences</em> 111 (5): 1748–53. <a href="https://doi.org/10.1073/pnas.1313601111">https://doi.org/10.1073/pnas.1313601111</a>.
</div>
</div></section></div> ]]></description>
  <category>physics</category>
  <category>philosophy</category>
  <guid>https://jianweicheong.codeberg.page/posts/life-conspiracy.html</guid>
  <pubDate>Tue, 30 Apr 2024 16:00:00 GMT</pubDate>
</item>
<item>
  <title>Haskell for numerical computation?</title>
  <link>https://jianweicheong.codeberg.page/posts/haskell-numerical.html</link>
  <description><![CDATA[ 




<section id="why-haskell" class="level1 page-columns page-full">
<h1>Why Haskell?</h1>
<p>Recently, I have been experimenting with <a href="https://en.wikipedia.org/wiki/Haskell">Haskell</a>, which I find to be quite enjoyable to program in thanks to its unique programming paradigm. So much so that I am considering using it for numerical computations for my work.</p>
<div class="page-columns page-full"><p></p><div class="no-row-height column-margin column-container"><span class="margin-aside"><strong>Side-effects</strong> refer to having an effect on the “outside world” such as printing a string to the terminal or writing to a file etc. Of course, we do want some side-effects for the program to do anything useful. In Haskell, side-effects are handled by <a href="https://en.wikipedia.org/wiki/Monad_(functional_programming)">monads</a>, which is a beast of a topic on its own. Monads allow the pure and impure (side-effects) to be clearly separated in Haskell.</span></div></div>
<p>Haskell (named after logician <a href="https://en.wikipedia.org/wiki/Haskell_Curry">Haskell Curry</a>) is a <a href="https://en.wikipedia.org/wiki/Purely_functional_programming">purely functional programming language</a> in the mathematical sense. Every function in Haskell simply takes an input and returns an output; nothing more, nothing less. It cannot mutate variables, and it cannot produce <strong><em>side-effects</em></strong>.</p>
<p>Purely functional programming languages are typically declarative, i.e., one does not write step-by-step instructions like imperative languages such as Python, C, C++, etc. For example, there are no loops in Haskell; one has to rely on recursion or <a href="https://en.wikipedia.org/wiki/Higher-order_function">higher-order functions</a> such as ‘fold’ and ‘scan’.</p>
<p>The two main reasons for my consideration to use it for numerical computations are:</p>
<ol type="1">
<li><p>Conciseness: Haskell’s declarative style greatly increase its conciseness. Prototyping and experimentation are frequent in computational research. Concise codes can reduce this programming time, and allow faster prototyping.</p></li>
<li><p>Safety and correctness: A small bug in numerical computations would not raise any error. Instead, it manifests in computed results or plots, masquerading as valid or even novel results. If these bugs are not identified, one might publish a paper with faux results.</p>
<ul>
<li>Haskell’s static typing can serve as an additional check to reduce bugs.</li>
<li>Its declarative approach means that codes are written in easily verifiable expressions—like in mathematics—rather than imperative statements that are often prone to human error.</li>
<li>Immutability and the absence of side-effects in functions can also reduce hidden bugs. For instance, the risk of a function modifying a variable without the coder’s knowledge is mitigated, and parallel computations are less likely to produce incorrect results due to variable mutations in <a href="https://en.wikipedia.org/wiki/Race_condition">race conditions</a>.</li>
</ul></li>
</ol>
<p>Though, I am neither a software developer nor am I computer science trained, and so some technical aspects of the language might escape me. Therefore, in this article, we will only examine some simple examples to compare traditional imperative languages like Python with the purely functional Haskell.</p>
</section>
<section id="some-examples" class="level1 page-columns page-full">
<h1>Some examples</h1>
<section id="sum-recursion" class="level2 page-columns page-full">
<h2 class="anchored" data-anchor-id="sum-recursion">Sum: recursion</h2>
<p>Ignoring the fact that the function <code>sum</code> is built-in in both Python and Haskell, let’s see how one might write a <code>sum</code> function that sum over all the elements in a list.</p>
<div class="tabset-margin-container"></div><div class="panel-tabset">
<ul class="nav nav-tabs"><li class="nav-item"><a class="nav-link active" id="tabset-1-1-tab" data-bs-toggle="tab" data-bs-target="#tabset-1-1" aria-controls="tabset-1-1" aria-selected="true" href="">Python</a></li><li class="nav-item"><a class="nav-link" id="tabset-1-2-tab" data-bs-toggle="tab" data-bs-target="#tabset-1-2" aria-controls="tabset-1-2" aria-selected="false" href="">Haskell</a></li></ul>
<div class="tab-content">
<div id="tabset-1-1" class="tab-pane active" aria-labelledby="tabset-1-1-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb1" style="background: #f1f3f5;"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb1-1"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">def</span> <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">sum</span>(xs):</span>
<span id="cb1-2">    sum_all <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span></span>
<span id="cb1-3">    <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">for</span> x <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">in</span> xs:</span>
<span id="cb1-4">        sum_all <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+=</span> x</span>
<span id="cb1-5">    <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">return</span> sum_all</span></code></pre></div></div>
</div>
<div id="tabset-1-2" class="tab-pane" aria-labelledby="tabset-1-2-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb2" style="background: #f1f3f5;"><pre class="sourceCode haskell code-with-copy"><code class="sourceCode haskell"><span id="cb2-1"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">sum</span><span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;"> ::</span> [<span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Float</span>] <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">-&gt;</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Float</span></span>
<span id="cb2-2"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">sum</span> [] <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span></span>
<span id="cb2-3"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">sum</span> (x<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span>xs) <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> x <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">sum</span> xs</span></code></pre></div></div>
</div>
</div>
</div>
<p>In the imperative Python example, we tell the program what to do step-by-step: first initialize an accumulator <code>sum_all</code> to 0, then loop through each elements in the list and add them to the accumulator.</p>
<div class="page-columns page-full"><p></p><div class="no-row-height column-margin column-container"><span class="margin-aside">One might notice that in Haskell, functions are applied without parenthesis, i.e., applying a function <code>foo</code> on a variable <code>x</code>, is typically <code>foo(x)</code>, but would be <code>foo x</code> in Haskell. The reason for this has something to do with the concept of “currying” which we will see later.</span></div></div>
<p>On the other hand, in the declarative Haskell example, we state that the sum of a list is simply the first element <code>x</code> plus the sum of the rest of the elements <code>xs</code>. This will then carry on recursively. As with all recursive functions, we require a base case to end the recursion, which is specified by <code>sum [] = 0</code>, which states to return 0 if the input to <code>sum</code> is an empty list. This ability to define the <code>sum</code> function twice with <code>sum [] = 0</code> and <code>sum (x:xs) = x + sum xs</code>, is simply <a href="https://en.wikipedia.org/wiki/Pattern_matching">pattern matching</a>.</p>
<p>Finally, Haskell is statically typed, i.e., the types of the inputs and outputs of the function can be specified which in our case is <code>sum :: [Float] -&gt; Float</code>, which states that the function <code>sum</code> takes in a list of floats <code>[Float]</code> and returns a single float <code>Float</code>. This type declaration looks similar to how one might write <img src="https://latex.codecogs.com/png.latex?f:%20X%20%5Cto%20Y"> mathematically for a function <img src="https://latex.codecogs.com/png.latex?y%20=%20f(x)">, where <img src="https://latex.codecogs.com/png.latex?x%20%5Cin%20X">, and <img src="https://latex.codecogs.com/png.latex?y%5Cin%20Y">.</p>
</section>
<section id="tensor-product-reduction-and-accumulation" class="level2 page-columns page-full">
<h2 class="anchored" data-anchor-id="tensor-product-reduction-and-accumulation">Tensor product: reduction and accumulation</h2>
<p>Here is a common example in quantum information/computation, where we have some quantum states that are complex matrices, and we want to perform a tensor product on them so that we can operate on them collectively.</p>
<blockquote class="blockquote">
<p>Given a list of matrices, the function <code>tensor_all</code> returns the tensor product of all the elements of the list in sequence, e.g., given the list of matrices <img src="https://latex.codecogs.com/png.latex?Ms%20=%20%5BA,%20B,%20C%5D">, <code>tensor_all(Ms)</code> should return their tensor product <img src="https://latex.codecogs.com/png.latex?A%5Cotimes%20B%5Cotimes%20C">.</p>
</blockquote>
<div class="tabset-margin-container"></div><div class="panel-tabset">
<ul class="nav nav-tabs"><li class="nav-item"><a class="nav-link active" id="tabset-2-1-tab" data-bs-toggle="tab" data-bs-target="#tabset-2-1" aria-controls="tabset-2-1" aria-selected="true" href="">Python</a></li><li class="nav-item"><a class="nav-link" id="tabset-2-2-tab" data-bs-toggle="tab" data-bs-target="#tabset-2-2" aria-controls="tabset-2-2" aria-selected="false" href="">Haskell</a></li></ul>
<div class="tab-content">
<div id="tabset-2-1" class="tab-pane active" aria-labelledby="tabset-2-1-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb3" style="background: #f1f3f5;"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb3-1"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">import</span> numpy <span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">as</span> np</span>
<span id="cb3-2"></span>
<span id="cb3-3"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">def</span> tensor_all(Ms):</span>
<span id="cb3-4">    tensor_prod <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span></span>
<span id="cb3-5">    <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">for</span> M <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">in</span> Ms:</span>
<span id="cb3-6">        tensor_prod <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> np.kron(tensor_prod, M)</span>
<span id="cb3-7"></span>
<span id="cb3-8">    <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">return</span> tensor_prod</span></code></pre></div></div>
</div>
<div id="tabset-2-2" class="tab-pane" aria-labelledby="tabset-2-2-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb4" style="background: #f1f3f5;"><pre class="sourceCode haskell code-with-copy"><code class="sourceCode haskell"><span id="cb4-1"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">import</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Numeric.LinearAlgebra</span></span>
<span id="cb4-2"></span>
<span id="cb4-3"><span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">tensor_all ::</span> [<span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Matrix</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">C</span>] <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">-&gt;</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Matrix</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">C</span></span>
<span id="cb4-4">tensor_all <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">foldl</span> kronecker <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span></span></code></pre></div></div>
</div>
</div>
</div>
<p>Imperatively in the Python example, we loop through each element of the list and apply the tensor product (kronecker product <code>np.kron</code>) element-by-element with an accumulator, i.e., in the first loop we have <img src="https://latex.codecogs.com/png.latex?(1)%20%5Cotimes%20A">, in the second loop we have <img src="https://latex.codecogs.com/png.latex?(1%5Cotimes%20A)%20%5Cotimes%20B">, in the third loop we have <img src="https://latex.codecogs.com/png.latex?(1%5Cotimes%20A%20%5Cotimes%20B)%20%5Cotimes%20C">, and so on and so forth, where our accumulator <code>tensor_prod</code> is the value in the parenthesis, is initialized as <img src="https://latex.codecogs.com/png.latex?1">, and is updated every loop.</p>
<div class="page-columns page-full"><p></p><div class="no-row-height column-margin column-container"><span class="margin-aside">Note that we have used the <code>hmatrix</code> library which is imported by <code>import Numeric.LinearAlgebra</code>. <a href="https://pechersky.github.io/haskell-numpy-docs/">This documentation</a> is a good resource for a comparison between <code>hmatrix</code> and Python’s <code>numpy</code>.</span></div></div>
<p>In Haskell, it should be obvious that we can perform a recursion like the case for <code>sum</code>. However, here we introduce the function <code>foldl</code>, which stands for “fold left”. <code>foldl</code> takes in a binary function (in this case <code>kronecker</code>), the initial accumulator (in this case <code>1</code>), and a list (e.g., <code>[a, b, c]</code>). It then returns <code>kronecker(kronecker(kronecker(1, a), b), c)</code>. There is also <code>foldr</code>, or “fold right”, which instead returns <code>kronecker(a, kronecker(b, kronecker(c, 1)))</code>.</p>
<div class="callout callout-style-default callout-tip callout-titled">
<div class="callout-header d-flex align-content-center collapsed" data-bs-toggle="collapse" data-bs-target=".callout-1-contents" aria-controls="callout-1" aria-expanded="false" aria-label="Toggle callout">
<div class="callout-icon-container">
<i class="callout-icon"></i>
</div>
<div class="callout-title-container flex-fill">
<span class="screen-reader-only">Tip</span>Point-free style programming
</div>
<div class="callout-btn-toggle d-inline-block border-0 py-1 ps-1 pe-0 float-end"><i class="callout-toggle"></i></div>
</div>
<div id="callout-1" class="callout-1-contents callout-collapse collapse">
<div class="callout-body-container callout-body">
<p>Note that in the Haskell example, we could have written the function as <code>tensor_all ms = foldl kronecker 1 ms</code>, where the input to the function, the list of matrices <code>ms</code>, is specified. However, since <code>ms</code> is simply the input to the corresponding function on both sides of the expression, we can omit it and simply write <code>tensor_all = foldl kronecker 1</code>, as if we “cancel” it out on both sides. This is referred to as <a href="https://en.wikipedia.org/wiki/Tacit_programming">tacit/point-free style programming</a>.</p>
</div>
</div>
</div>
<p>If instead, we want to “accumulate” the result of each loop into a list, i.e., we want the function to return <img src="https://latex.codecogs.com/png.latex?%5BA,%20A%20%5Cotimes%20B,%20A%5Cotimes%20B%5Cotimes%20C%5D">, then, in the Python example we have to append the accumulator <code>tensor_prod</code> in each loop into a list. On the other hand, in the Haskell example, we can simply replace <code>foldl</code> with <code>scanl</code>.</p>
<div class="tabset-margin-container"></div><div class="panel-tabset">
<ul class="nav nav-tabs"><li class="nav-item"><a class="nav-link active" id="tabset-3-1-tab" data-bs-toggle="tab" data-bs-target="#tabset-3-1" aria-controls="tabset-3-1" aria-selected="true" href="">Python</a></li><li class="nav-item"><a class="nav-link" id="tabset-3-2-tab" data-bs-toggle="tab" data-bs-target="#tabset-3-2" aria-controls="tabset-3-2" aria-selected="false" href="">Haskell</a></li></ul>
<div class="tab-content">
<div id="tabset-3-1" class="tab-pane active" aria-labelledby="tabset-3-1-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb5" style="background: #f1f3f5;"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb5-1"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">import</span> numpy <span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">as</span> np</span>
<span id="cb5-2"></span>
<span id="cb5-3"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">def</span> tensor_all(Ms):</span>
<span id="cb5-4">    tensor_prod <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span></span>
<span id="cb5-5">    tensor_prod_list <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> []</span>
<span id="cb5-6">    <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">for</span> M <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">in</span> Ms:</span>
<span id="cb5-7">        tensor_prod <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> np.kron(tensor_prod, M)</span>
<span id="cb5-8">        tensor_prod_list.append(tensor_prod)</span>
<span id="cb5-9"></span>
<span id="cb5-10">    <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">return</span> tensor_prod_list</span></code></pre></div></div>
</div>
<div id="tabset-3-2" class="tab-pane" aria-labelledby="tabset-3-2-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb6" style="background: #f1f3f5;"><pre class="sourceCode haskell code-with-copy"><code class="sourceCode haskell"><span id="cb6-1"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">import</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Numeric.LinearAlgebra</span></span>
<span id="cb6-2"></span>
<span id="cb6-3"><span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">tensor_all ::</span> [<span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Matrix</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">C</span>] <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">-&gt;</span> [<span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Matrix</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">C</span>]</span>
<span id="cb6-4">tensor_all <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">scanl</span> kronecker <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span></span></code></pre></div></div>
</div>
</div>
</div>
<p>Folds and scans are part and parcel of a functional programmer, and are common in functional programming languages, as well as <strong><em>array programming languages</em></strong>. Typically, they are referred to as reduction and accumulation, e.g., in the array programming languages NumPy, Julia, and R, folds and scans are called <code>reduce</code> and <code>accumulate</code>.</p>

<div class="no-row-height column-margin column-container"><div class="">
<div class="quarto-figure quarto-figure-center">
<figure class="figure">
<p><img src="https://user-images.githubusercontent.com/36027403/193055974-8d8008a2-a1e6-4bd4-8ecc-7624fa611f25.png" class="imghover img-fluid figure-img"></p>
<figcaption>Image credit: <a href="https://codereport.github.io/">code_report</a></figcaption>
</figure>
</div>
<p>If you would like to know more about <strong><em>array languages</em></strong>, and how they compare to functional languages like Haskell, I highly recommend the youtube channel <a href="https://www.youtube.com/c/codereport">code_report</a>.</p>
</div></div></section>
<section id="collision-model-currying-and-laziness" class="level2 page-columns page-full">
<h2 class="anchored" data-anchor-id="collision-model-currying-and-laziness">Collision model: currying and laziness</h2>
<p>Let’s consider a less simple example to reveal more of Haskell’s features:</p>
<blockquote class="blockquote">
<p>A superconducting qubit <img src="https://latex.codecogs.com/png.latex?%5Crho"> (<code>rho</code>) is sensitive to heat, and is continuously undergoing a thermalization process. We can model this thermalization process with a collision model, where the qubit evolves via repeated “collisions” with different qubits at each discrete time step for <img src="https://latex.codecogs.com/png.latex?n"> (<code>n</code>) number of time steps. The collision is implemented by the function <code>collision</code>. We want to determine how the von Neumann entropy of <img src="https://latex.codecogs.com/png.latex?%5Crho">, <img src="https://latex.codecogs.com/png.latex?S(%5Crho)">, changes in each collision up till <img src="https://latex.codecogs.com/png.latex?n"> time steps. The calculation of the von Neumann entropy is implemented by the function <code>entropy</code>.</p>
</blockquote>
<div class="page-columns page-full"><p></p><div class="no-row-height column-margin column-container"><span class="margin-aside">For completeness, the function <code>collision</code> performs the operation of <img src="https://latex.codecogs.com/png.latex?%0A%5Crho_%7Bt+1%7D%0A=%20%5Csum_i%20K_i%20%5Crho_t%20K_i%5E%5Cdagger,%0A"> where <img src="https://latex.codecogs.com/png.latex?K_i"> are Kraus operators, and the function <code>entropy</code> computes <img src="https://latex.codecogs.com/png.latex?%0AS%5Cleft(%5Crho_t%5Cright)%20=%20-%5Cmathrm%7BTr%7D%20%5Cleft(%5Crho_t%20%5Clog%20%5Crho_t%5Cright).%0A"> Therefore, what we want is simply the list of <img src="https://latex.codecogs.com/png.latex?%5Csmall%5BS(%5Crho_0),%20S(%5Crho_1),%20%5Cldots,%20S(%5Crho_%7Bn-1%7D)%5D"></span></div></div>
<p>This might look complicated if one does not know quantum mechanics. However, if we assume that the collision and entropy calculations are done for us in the functions <code>collision</code> and <code>entropy</code>, then we simply want a list of <code>n</code> entropy values, i.e., <code>entropy_list = [entropy(rho), entropy(collision(rho)), entropy(collision(collision(rho))),...]</code>.</p>
<div class="tabset-margin-container"></div><div class="panel-tabset">
<ul class="nav nav-tabs"><li class="nav-item"><a class="nav-link active" id="tabset-4-1-tab" data-bs-toggle="tab" data-bs-target="#tabset-4-1" aria-controls="tabset-4-1" aria-selected="true" href="">Python</a></li><li class="nav-item"><a class="nav-link" id="tabset-4-2-tab" data-bs-toggle="tab" data-bs-target="#tabset-4-2" aria-controls="tabset-4-2" aria-selected="false" href="">Haskell</a></li></ul>
<div class="tab-content">
<div id="tabset-4-1" class="tab-pane active" aria-labelledby="tabset-4-1-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb7" style="background: #f1f3f5;"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb7-1"><span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">import</span> scipy <span class="im" style="color: #00769E;
background-color: null;
font-style: inherit;">as</span> sp</span>
<span id="cb7-2"></span>
<span id="cb7-3"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">def</span> collision(ks, rho):</span>
<span id="cb7-4">    <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">return</span> <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">sum</span>([k <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">@</span> rho <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">@</span> k.conj().T <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">for</span> k <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">in</span> ks])</span>
<span id="cb7-5"></span>
<span id="cb7-6"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">def</span> entropy(rho):</span>
<span id="cb7-7">    <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">return</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span> np.trace(rho <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">@</span> sp.linalg.logm(rho))</span>
<span id="cb7-8"></span>
<span id="cb7-9"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">def</span> calc_entropies(ks, rho, n):</span>
<span id="cb7-10">    entropy_list <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> []</span>
<span id="cb7-11">    <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">for</span> _ <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">in</span> <span class="bu" style="color: null;
background-color: null;
font-style: inherit;">range</span>(n):</span>
<span id="cb7-12">        rho_entropy <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> entropy(rho)</span>
<span id="cb7-13">        entropy_list.append(rho_entropy)</span>
<span id="cb7-14">        rho <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> collision(ks, rho)</span>
<span id="cb7-15"></span>
<span id="cb7-16">    <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">return</span> entropy_list</span></code></pre></div></div>
</div>
<div id="tabset-4-2" class="tab-pane" aria-labelledby="tabset-4-2-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb8" style="background: #f1f3f5;"><pre class="sourceCode haskell code-with-copy"><code class="sourceCode haskell"><span id="cb8-1"><span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">collision ::</span> [<span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Matrix</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">C</span>] <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">-&gt;</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Matrix</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">C</span> <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">-&gt;</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Matrix</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">C</span></span>
<span id="cb8-2">collision ks rho <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">sum</span> [k <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;&gt;</span> rho <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;&gt;</span> tr k <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|</span> k <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> ks]</span>
<span id="cb8-3"></span>
<span id="cb8-4"><span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">entropy ::</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Matrix</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">C</span> <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">-&gt;</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Float</span></span>
<span id="cb8-5">entropy rho <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span> trace (rho <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;&gt;</span> logm rho)</span>
<span id="cb8-6">  <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">where</span></span>
<span id="cb8-7">    logm <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> matFunc <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">log</span></span>
<span id="cb8-8"></span>
<span id="cb8-9"><span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">calc_entropies ::</span> [<span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Matrix</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">C</span>] <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">-&gt;</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Matrix</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">C</span> <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">-&gt;</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Int</span> <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">-&gt;</span> [<span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Float</span>]</span>
<span id="cb8-10">calc_entropies ks rho n <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">map</span> entropy rho_list</span>
<span id="cb8-11">  <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">where</span></span>
<span id="cb8-12">    rho_list <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">take</span> n (<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">iterate</span> (collision ks) rho)</span></code></pre></div></div>
</div>
</div>
</div>
<div class="callout callout-style-default callout-note callout-titled">
<div class="callout-header d-flex align-content-center collapsed" data-bs-toggle="collapse" data-bs-target=".callout-2-contents" aria-controls="callout-2" aria-expanded="false" aria-label="Toggle callout">
<div class="callout-icon-container">
<i class="callout-icon"></i>
</div>
<div class="callout-title-container flex-fill">
<span class="screen-reader-only">Note</span>Extra things to note for the Haskell example
</div>
<div class="callout-btn-toggle d-inline-block border-0 py-1 ps-1 pe-0 float-end"><i class="callout-toggle"></i></div>
</div>
<div id="callout-2" class="callout-2-contents callout-collapse collapse">
<div class="callout-body-container callout-body">
<ul>
<li><p>Similar to Python, Haskell has list comprehension, which has a mathematical syntax as seen in the function <code>collision</code>, e.g.,</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb9" style="background: #f1f3f5;"><pre class="sourceCode haskell code-with-copy"><code class="sourceCode haskell"><span id="cb9-1">example <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> [p <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> q <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">|</span> p <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> ps, q <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> qs, p <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;=</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">5</span>, q <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">/=</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>]</span></code></pre></div></div>
<p>would give a list of the set <img src="https://latex.codecogs.com/png.latex?%5C%7Bpq%5C%20%7C%5C%20p%20%5Cin%20P,%5C%20q%20%5Cin%20Q,%5C%20p%20%5Cgeq%205,%5C%20q%20%5Cneq%200%20%5C%7D."></p>
<p>On the other hand, Python’s list comprehension is less concise:</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb10" style="background: #f1f3f5;"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb10-1">example <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> [p <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> q <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">for</span> p <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">in</span> ps <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">for</span> q <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">in</span> qs <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">if</span> p <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;=</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">5</span> <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">and</span> q <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">!=</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>]</span></code></pre></div></div></li>
<li><p>In the <code>entropy</code> and <code>calc_entropies</code> functions, we have used the <code>where</code> keyword, which allows us to break a function into smaller constituents, e.g.,</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb11" style="background: #f1f3f5;"><pre class="sourceCode haskell code-with-copy"><code class="sourceCode haskell"><span id="cb11-1">exampleFunc a b c <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> n <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">/</span> d</span>
<span id="cb11-2">  <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">where</span></span>
<span id="cb11-3">    n <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> x <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> y</span>
<span id="cb11-4">    d <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> x <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span> y</span>
<span id="cb11-5">    x <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> a <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> b <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> c</span>
<span id="cb11-6">    y <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> a <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> b <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> c</span></code></pre></div></div>
<p>Alternatively, we can also use the <code>let</code> and <code>in</code> keyword:</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb12" style="background: #f1f3f5;"><pre class="sourceCode haskell code-with-copy"><code class="sourceCode haskell"><span id="cb12-1">exampleFunc a b c <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span></span>
<span id="cb12-2">  <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">let</span></span>
<span id="cb12-3">    n <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> x <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> y</span>
<span id="cb12-4">    d <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> x <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span> y</span>
<span id="cb12-5">    x <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> a <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> b <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> c</span>
<span id="cb12-6">    y <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> a <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> b <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">*</span> c</span>
<span id="cb12-7">  <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">in</span></span>
<span id="cb12-8">    n <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">/</span> d</span></code></pre></div></div></li>
</ul>
</div>
</div>
</div>
<p>The Python example, being imperative, should be self-explanatory. Instead, we are interested in the Haskell example, specifically the <code>calc_entropies</code> function, which might look arcane if one does not know common functional programming functions such as <code>map</code>, <code>take</code>, and <code>iterate</code>. Let’s look at this line-by-line.</p>
<section id="currying" class="level3">
<h3 class="anchored" data-anchor-id="currying">Currying</h3>
<p>The very first line of the <code>calc_entropies</code> function is the type signature of:</p>
<p><img src="https://latex.codecogs.com/png.latex?%0A%5Cmathtt%7Bcalc%5C_entropies%20::%20%5BMatrix%5C%20C%5D%5C%20-%3E%20Matrix%5C%20C%5C%20-%3E%20Int%5C%20-%3E%20%5BFloat%5D%7D%0A"></p>
<p>which states that the function <code>calc_entropies</code> takes in a list of complex matrix, a complex matrix, and an integer, and returns a list of floats. The list of complex matrix refers to the Kraus operators <img src="https://latex.codecogs.com/png.latex?K_i"> used in the <code>collision</code> function, the complex matrix refers to the qubit <code>rho</code> or <img src="https://latex.codecogs.com/png.latex?%5Crho">, the integer refers to the number of collisions <code>n</code>, while the list of floats refers to the output <code>entropy_list</code>.</p>
<p>One might notice that <code>[Matrix C] -&gt; Matrix C -&gt; Int -&gt; [Float]</code> doesn’t seem to make a clear distinction between inputs and outputs. This has to do with the concept of <strong><em>partial function application</em></strong>, or <a href="https://en.wikipedia.org/wiki/Currying"><strong><em>currying</em></strong></a> (also named after Haskell Curry).</p>
<p>Implicitly, what’s happening is the following:</p>
<p><img src="https://latex.codecogs.com/png.latex?%0A%5Cmathtt%7Bcalc%5C_entropies%20::%20%5BMatrix%5C%20C%5D%5C%20-%3E%5C%20%7D%20%5Cunderbrace%7B%5Cmathtt%7BMatrix%5C%20C%5C%20-%3E%20Int%5C%20-%3E%20%5BFloat%5D%7D%7D_%5Ctext%7Bfunction%20$f$%7D%0A"> <img src="https://latex.codecogs.com/png.latex?%0Af%5C%20%5Cmathtt%7B::%20Matrix%5C%20C%5C%20-%3E%5C%20%7D%20%5Cunderbrace%7B%5Cmathtt%7BInt%5C%20-%3E%20%5BFloat%5D%7D%7D_%5Ctext%7Bfunction%20$g$%7D%0A"> <img src="https://latex.codecogs.com/png.latex?%0Ag%5C%20%5Cmathtt%7B::%20Int%5C%20-%3E%20%5BFloat%5D%7D%0A"></p>
<p>where the function <code>calc_entropies</code> takes in <code>[Matrix C]</code> as its input, and returns a function <img src="https://latex.codecogs.com/png.latex?f"> as its output. The function <img src="https://latex.codecogs.com/png.latex?f"> then takes in <code>Matrix C</code> as the input and returns another function <img src="https://latex.codecogs.com/png.latex?g"> as the output. Finally, the function <img src="https://latex.codecogs.com/png.latex?g"> takes in <code>Int</code> as the input and returns <code>[Float]</code> as the final output. This means that functions in Haskell are indeed pure in that they only take in one input and return one output, and any functions that appear to take in multiple inputs are in fact taking in only one input and returning a function that also takes in one input as its output, and so on and so forth. This is called currying. Because of this, we can also apply the functions “partially”, for example:</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb13" style="background: #f1f3f5;"><pre class="sourceCode haskell code-with-copy"><code class="sourceCode haskell"><span id="cb13-1">f a b <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> a <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> b</span>
<span id="cb13-2">g <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> f <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">5</span></span></code></pre></div></div>
<p>where we only provided one argument to <code>f</code> when it is expecting two, to create a new function <code>g</code>. This means that <code>f 5 10</code> and <code>g 10</code> would give the same output of <code>15</code>. Currying or partial function application can be a powerful tool for the abstraction and expressiveness of your code.</p>
<div class="callout callout-style-default callout-tip callout-titled">
<div class="callout-header d-flex align-content-center collapsed" data-bs-toggle="collapse" data-bs-target=".callout-3-contents" aria-controls="callout-3" aria-expanded="false" aria-label="Toggle callout">
<div class="callout-icon-container">
<i class="callout-icon"></i>
</div>
<div class="callout-title-container flex-fill">
<span class="screen-reader-only">Tip</span>Example of the usefulness of currying
</div>
<div class="callout-btn-toggle d-inline-block border-0 py-1 ps-1 pe-0 float-end"><i class="callout-toggle"></i></div>
</div>
<div id="callout-3" class="callout-3-contents callout-collapse collapse">
<div class="callout-body-container callout-body">
<p>Unitary operations are common in quantum mechanics, and are how quantum computers perform computations on quantum states or qubits. For example, the unitary <img src="https://latex.codecogs.com/png.latex?U"> can act on a quantum state <img src="https://latex.codecogs.com/png.latex?%7C%5Cpsi%5Crangle"> as follows: <img src="https://latex.codecogs.com/png.latex?%0A%7C%5Cpsi'%5Crangle%20=%20U%20%7C%5Cpsi%5Crangle.%0A"></p>
<p>We can define a function <code>unitary_oper</code> to implement this.</p>
<div class="tabset-margin-container"></div><div class="panel-tabset">
<ul class="nav nav-tabs"><li class="nav-item"><a class="nav-link active" id="tabset-5-1-tab" data-bs-toggle="tab" data-bs-target="#tabset-5-1" aria-controls="tabset-5-1" aria-selected="true" href="">Python</a></li><li class="nav-item"><a class="nav-link" id="tabset-5-2-tab" data-bs-toggle="tab" data-bs-target="#tabset-5-2" aria-controls="tabset-5-2" aria-selected="false" href="">Haskell</a></li></ul>
<div class="tab-content">
<div id="tabset-5-1" class="tab-pane active" aria-labelledby="tabset-5-1-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb14" style="background: #f1f3f5;"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb14-1"><span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">def</span> unitary_oper(U, ket):</span>
<span id="cb14-2">    <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">return</span> U <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">@</span> ket</span></code></pre></div></div>
</div>
<div id="tabset-5-2" class="tab-pane" aria-labelledby="tabset-5-2-tab">
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb15" style="background: #f1f3f5;"><pre class="sourceCode haskell code-with-copy"><code class="sourceCode haskell"><span id="cb15-1"><span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">unitary_oper ::</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Matrix</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">C</span> <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">-&gt;</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Matrix</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">C</span> <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">-&gt;</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">Matrix</span> <span class="dt" style="color: #AD0000;
background-color: null;
font-style: inherit;">C</span></span>
<span id="cb15-2">unitary_oper u ket <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> u <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&lt;&gt;</span> ket</span></code></pre></div></div>
</div>
</div>
</div>
<p>There are many common unitaries <img src="https://latex.codecogs.com/png.latex?U"> used in quantum computations, referred to as <a href="https://en.wikipedia.org/wiki/Quantum_logic_gate">quantum logic gates</a>. For example there are the Pauli X gate and the Hadamard gate which are defined as <img src="https://latex.codecogs.com/png.latex?%0AX%20=%0A%5Cbegin%7Bpmatrix%7D%0A0%20&amp;%201%20%5C%5C%0A1%20&amp;%200%0A%5Cend%7Bpmatrix%7D,%5Cquad%0AH%20=%20%5Cfrac%7B1%7D%7B%5Csqrt%7B2%7D%7D%0A%5Cbegin%7Bpmatrix%7D%0A1%20&amp;%201%20%5C%5C%0A1%20&amp;%20-1%0A%5Cend%7Bpmatrix%7D.%0A"></p>
<p>Currying or partial function application allows us to easily define these new gate operations on top of the existing <code>unitary_oper</code> function:</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb16" style="background: #f1f3f5;"><pre class="sourceCode haskell code-with-copy"><code class="sourceCode haskell"><span id="cb16-1">pauli_x <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> unitary_oper (<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;&lt;</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>) [<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>]</span>
<span id="cb16-2">hadamard <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> unitary_oper (cmap (<span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">/</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">sqrt</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>) (<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span><span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;&lt;</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>) [<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>, <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>])</span></code></pre></div></div>
<p>We can then easily apply the Pauli X and Hadamard gates on a qubit <img src="https://latex.codecogs.com/png.latex?%7C0%5Crangle%20=%20%5B1,%200%5D%5ET"> with <code>pauli_x (2&gt;&lt;1) [1, 0]</code> and <code>hadamard (2&gt;&lt;1) [1, 0]</code>. This can make the code more concise and clearer.</p>
</div>
</div>
</div>
</section>
<section id="lazy-evaluation" class="level3">
<h3 class="anchored" data-anchor-id="lazy-evaluation">Lazy evaluation</h3>
<p>Moving on to the function itself:</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb17" style="background: #f1f3f5;"><pre class="sourceCode haskell code-with-copy"><code class="sourceCode haskell"><span id="cb17-1">calc_entropies ks rho n <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">map</span> entropy rho_list</span>
<span id="cb17-2">  <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">where</span></span>
<span id="cb17-3">    rho_list <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">take</span> n (<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">iterate</span> (collision ks) rho)</span></code></pre></div></div>
<p>The first line features the function <code>map</code>, which is the main tool of the functional programmer. It simply apply a function to every element of a list:</p>
<p><img src="https://latex.codecogs.com/png.latex?%0A%5Cunderbrace%7B%5Cmathtt%7Bmap%5Cquad%20entropy%7D%7D_%5Ctext%7Bapply%20entropy%20function%20to%20each%20element%20of%7D%5Cquad%20%5Cmathtt%7Brho%5C_list%7D%0A"></p>
<p>and has the corresponding Python code of</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb18" style="background: #f1f3f5;"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb18-1">entropy_list <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> [entropy(rho) <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">for</span> rho <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">in</span> rho_list]</span>
<span id="cb18-2"></span>
<span id="cb18-3"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># or alternatively</span></span>
<span id="cb18-4">entropy_list <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> []</span>
<span id="cb18-5"><span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">for</span> rho <span class="kw" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">in</span> rho_list:</span>
<span id="cb18-6">    entropy_list.append(entropy(rho))</span></code></pre></div></div>
<p>Finally, in the last line we have</p>
<p><img src="https://latex.codecogs.com/png.latex?%5Cmathtt%7Bwhere%5Cquad%20rho%5C_list%7D%20="> <img src="https://latex.codecogs.com/png.latex?%0A%5Cunderbrace%7B%5Cmathtt%7Btake%5Cquad%20n%7D%7D_%5Ctext%7Btake%20first%20$n$%20of%7D%5Cquad%20%5Cunderbrace%7B%5Cmathtt%7B(iterate%5Cquad%20(collision%5Cquad%20ks)%5Cquad%20rho)%7D%7D_%5Ctext%7Breturns%20an%20infinite-sized%20list%20of%20%5Brho,%20collision(rho),%20collision(collision(rho)),...%5D%7D%0A"> which gives <code>rho_list = [rho, collision(rho), collision(collison(rho)),...]</code> up till <code>n</code> number of elements in the list.</p>
<p>Also note that we can reduce the number of parenthesis by using <code>$</code> instead, e.g., <code>rho_list = take n $ iterate (collision ks) rho</code>.</p>
<p>It might be quite surprising that the <code>iterate</code> function returns an infinite-sized list. For example, <code>iterate add1 10</code> gives <code>[11, 12, 13, 14, 15, ...]</code> to infinity, where the function <code>add1</code> is applied on <code>10</code> ad infinitum. If we were to write the corresponding Python code for <code>rho_list = take n (iterate (collision ks) rho)</code>, it might look something like</p>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb19" style="background: #f1f3f5;"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb19-1">rhos <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> []</span>
<span id="cb19-2"><span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">while</span> <span class="va" style="color: #111111;
background-color: null;
font-style: inherit;">True</span>:</span>
<span id="cb19-3">    rhos.append(rho)</span>
<span id="cb19-4">    rho <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> collision(ks, rho)</span>
<span id="cb19-5"></span>
<span id="cb19-6">rho_list <span class="op" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">=</span> rhos[:n]</span></code></pre></div></div>
<p>which runs forever and result in a list of infinite size. However, this is not a problem in Haskell, thanks to the fact that Haskell is <a href="https://en.wikipedia.org/wiki/Lazy_evaluation"><strong><em>lazy-evaluated</em></strong></a>. This means that Haskell only evaluates values when required, and so since we only require the first <code>n</code> elements of the infinite list, Haskell only evaluate that first <code>n</code> elements.</p>
<p>If this Python code is lazy-evaluated like in Haskell, one might imagine that the program looks ahead and saw that it only require the first <code>n</code> elements as stated in <code>rho_list = rhos[:n]</code>, and so it stops the <code>while True</code> loop after only running <code>n</code> times.</p>
</section>
</section>
</section>
<section id="conclusion" class="level1 page-columns page-full">
<h1>Conclusion</h1>
<div class="page-columns page-full"><p></p><div class="no-row-height column-margin column-container"><span class="margin-aside">Websites like <a href="https://hoogle.haskell.org/">Hoogle</a> and <a href="https://hoogletranslate.com/">Hoogle translate</a> can greatly ease the learning curve of Haskell.</span></div></div>
<p>Overall, I find Haskell’s expressions to be much more concise and clearer than imperative statements in Python or most ALGOL family of languages. Of course, if one is unfamiliar with higher-order functions such as <code>map</code>, <code>iterate</code>, <code>foldl</code>, or even <a href="https://en.wikipedia.org/wiki/Combinatory_logic">combinators</a> etc., then Haskell definitely look less concise and less clear, but this is just the standard experience of learning a new language.</p>
<p>It is also often said that learning Haskell will change the way you write code even in other programming languages, and will make you a better programmer in general, e.g., favoring small and pure composable functions, preferring immutable data structures, or embracing “higher-order thinking”. In other words, learning Haskell is more than just picking up new syntax; it reshapes how you approach programming.</p>


</section>

<a onclick="window.scrollTo(0, 0); return false;" id="quarto-back-to-top"><i class="bi bi-arrow-up"></i> Back to top</a> ]]></description>
  <category>software</category>
  <category>programming</category>
  <guid>https://jianweicheong.codeberg.page/posts/haskell-numerical.html</guid>
  <pubDate>Mon, 04 Sep 2023 16:00:00 GMT</pubDate>
</item>
<item>
  <title>Why I recommend Fedora for new Linux users</title>
  <link>https://jianweicheong.codeberg.page/posts/why-fedora.html</link>
  <description><![CDATA[ 




<p>Here, I provide a list of reasons on why I think Fedora is a good starting point for new Linux users. It comes down to a single advantage: good defaults.</p>
<section id="semi-rolling-release" class="level1">
<h1>Semi-rolling release</h1>
<p>When you use your computer, here’s what you might expect:</p>
<ol type="1">
<li>All user programs are the latest versions.</li>
<li>All system programs needed for the operating system are the latest versions.</li>
<li>But you don’t want your experience to keep changing.</li>
</ol>
<p>This is reasonable; you want frequent updates because you want the latest features or security and bug fixes. But at the same time, you don’t want your desktop to completely change after an update (think Windows 7 to 8, or 10 to 11).</p>
<p>Fedora’s semi-rolling release cycle abides by these: you get frequent program updates (including kernel updates), but some packages like the desktop environment GNOME, or major software decisions (like moving to a new technology), are only updated in a major version release that the user must initiate.</p>
<p>This is a good middle ground between the two extremes of rolling-release where everything gets updated, and “stable”-release where you are often using programs that have not been updated in years.</p>
</section>
<section id="wayland" class="level1">
<h1>Wayland</h1>
<p>Most Linux distros use <code>X11</code> as its windowing system (a program that handles how windows are drawn on your screen). However, <code>X11</code> does not have GUI isolation, allowing different programs to interact with one another. A malicious program can then record your screen, or log the keys you typed in your browser while you are logging in to your bank.</p>
<p><code>Wayland</code> is the successor to <code>X11</code>, and is developed with security in mind, isolating each window from one another.</p>
<blockquote class="blockquote">
<p>Unlike X, the Wayland input stack doesn’t allow applications to snoop on the input of other programs (preserving confidentiality), to generate input events that appear to come from the user (preserving input integrity), or to capture all the input events to the exclusion of the user’s application (preserving availability).</p>
<p>— <a href="https://lwn.net/Articles/589147/">Jake Edge, LWN.net</a></p>
</blockquote>
</section>
<section id="swap-on-zram" class="level1">
<h1>Swap on ZRAM</h1>
<p>Linux systems offer a “swap” space, where memory on the RAM can be copied to disk space, freeing the RAM in the process. Swap is an <a href="https://chrisdown.name/2018/01/02/in-defence-of-swap.html">important part</a> of a Linux operating system. However, sensitive information stored on RAM can be copied to disk, risking potential security issues. <code>ZRAM</code> solves these problems by having the swap space remain in the RAM with compression, ensuring that no sensitive information will be stored on disk.</p>
</section>
<section id="btrfs" class="level1">
<h1>Btrfs</h1>
<p>Most Linux distros have traditionally used the <code>ext4</code> filesystem. <code>btrfs</code> is a newer filesystem that offers advantages over <code>ext4</code>:</p>
<ul>
<li>Compression leads to less disk space used.</li>
<li>Compression also extends the lifespan of flash-based devices (SSDs, etc.).</li>
<li>Less likely to have data corruption.</li>
<li>Provides snapshots that allows rollback when system breaks.</li>
</ul>
<blockquote class="blockquote">
<p>For laptop and workstation installs of Fedora, we want to provide file system features to users in a transparent fashion. We want to add new features, while reducing the amount of expertise needed to deal with situations like running out of disk space. Btrfs is well adapted to this role by design philosophy, let’s make it the default.</p>
<p>…</p>
<p>The Btrfs community has users that have been using it for most of the past decade at scale. It’s been the default on openSUSE (and SUSE Linux Enterprise) since 2014, and Facebook has been using it for all their OS and data volumes, in their data centers, for almost as long. Btrfs is a mature, well-understood, and battle-tested file system, used on both desktop/container and server/cloud use-cases.</p>
<p>— <a href="https://fedoraproject.org/wiki/Changes/BtrfsByDefault">Fedora Project wiki, Btrfs changes proposal</a></p>
</blockquote>
</section>
<section id="selinux-and-firewalld" class="level1">
<h1>SELinux and firewalld</h1>
<p>Security Enhanced Linux (<code>SELinux</code>) provides Mandatory Access Control (MAC).</p>
<blockquote class="blockquote">
<p>In computer security, mandatory access control (MAC) refers to a type of access control by which the operating system or database constrains the ability of a subject or initiator to access or generally perform some sort of operation on an object or target.</p>
<p>— <a href="https://en.wikipedia.org/wiki/Mandatory_access_control">Wikipedia</a></p>
</blockquote>
<blockquote class="blockquote">
<p>Security Enhanced Linux (SELinux) provides an additional layer of system security. SELinux fundamentally answers the question: May &lt;subject&gt; do &lt;action&gt; to &lt;object&gt;?, for example: May a web server access files in users’ home directories?</p>
<p>— <a href="https://docs.fedoraproject.org/en-US/quick-docs/getting-started-with-selinux/">Fedora docs</a></p>
</blockquote>
<p>Fedora comes with good default policies for <code>SELinux</code>, as well as good default firewall rules with <code>firewalld</code>.</p>
</section>
<section id="flatpak-enabled" class="level1">
<h1>Flatpak enabled</h1>
<p><code>Flatpak</code> is supposed to be a new and universal way of installing programs. It is now available on almost all major distros and is also the most popular way of installing programs on the Steam Deck.</p>
<p>Ubuntu on the other hand, while often recommended to beginners, decided to <a href="https://discourse.ubuntu.com/t/ubuntu-flavor-packaging-defaults/34061?u=d0od">drop Flatpak</a> in favor of <code>snap</code>, which is their own alternative that is much less popular. <code>snap</code> packages <a href="https://askubuntu.com/questions/1185091/why-apt-package-chromium-browser-installs-snap-package-instead">masquerade as traditional packages</a>, which can cause confusion for new users. They also <a href="https://forum.snapcraft.io/t/sudo-snap-install-or-just-snap-install/3909">encourage users to sign up an Ubuntu One account</a> just to install <code>snap</code> packages without root.</p>
</section>
<section id="speed-of-adopting-new-technologies" class="level1">
<h1>Speed of adopting new technologies</h1>
<p>Fedora tends to adopt new technologies at a fast pace, such as <code>Wayland</code>, swap on <code>ZRAM</code>, <code>Flatpak</code>, <code>PipeWire</code>, <code>btrfs</code> etc. New technologies tends to offer improvements in terms of usability and security.</p>
<p>For example, Fedora adopted <code>Wayland</code> in 2016, but Ubuntu only done so in 2021. Even Red Hat Enterprise Linux (RHEL) and Debian adopted <code>Wayland</code> in 2019, earlier than Ubuntu, despite both being extremely conservative distros.</p>
</section>
<section id="whats-next" class="level1">
<h1>What’s next?</h1>
<p>If you just want a usable system with no interest in learning more about the Linux ecosystem, then I believe Fedora is the ideal choice. However, I encourage people to venture out on their own in their Linux journey. One should try out different Linux distros and explore alternatives to learn more about the Linux ecosystem.</p>
<p>Eventually, as you gain more understanding of the Linux ecosystem, you will develop your own opinions on things. For example, you might be <a href="https://flatkill.org/2020/">against Flatpak</a>, or perhaps you might think that <a href="https://nosystemd.org/">systemd has major problems</a>. Regardless, the greatest strength of the Linux ecosystem is the power to make your own choices on what software and technologies you prefer.</p>


</section>

<a onclick="window.scrollTo(0, 0); return false;" id="quarto-back-to-top"><i class="bi bi-arrow-up"></i> Back to top</a> ]]></description>
  <category>software</category>
  <category>linux</category>
  <guid>https://jianweicheong.codeberg.page/posts/why-fedora.html</guid>
  <pubDate>Fri, 31 Mar 2023 16:00:00 GMT</pubDate>
</item>
</channel>
</rss>
