Posts tagged "Racket"urn:www-greghendershott-com:Racket2020-02-24T00:00:00ZThe Big Switcheroourn:www-greghendershott-com:-2020-02-the-big-switcheroo.html2020-02-24T00:00:00Z2020-02-24T00:00:00ZGreg Hendershott
<div>
<article>
<header>
<h1>The Big Switcheroo</h1>
<p class="date-and-tags">
<time datetime="2020-02-24" pubdate="true">2020-02-24</time> :: <span class="tags"><a href="/tags/Racket.html">Racket</a>, <a href="/tags/Emacs.html">Emacs</a></span></p></header>
<p>In my <a href="/2020/02/using-drracket-check-syntax-in-racket-mode.html">previous post</a> I talked about how Racket Mode now will often want the back end command server, before actually needing a live REPL — but it has to start a REPL anyway. This was bothering me. So I went on to address that. Doing so entailed reversing the I/O model for the back end. As a bonus, that set up fairly short strokes to supporting multiple REPLs.</p><!-- more-->
<p>In the beginning, Racket Mode was a thin wrapper around <code>xrepl</code>. As a result, it used a <code>comint-mode</code> buffer to start Racket, and the REPL stdin/stdout was connected to the <code>comint-mode</code> buffer. The back end also started a little TCP server; connecting to that established the I/O for commands.</p>
<p>This was mostly fine, especially when <code>racket-run</code> tended to be the first command you would use.</p>
<p>However it might take a second or two for the TCP server to be ready to accept connections. As a result, the front end either needed to block (boo!) or use an Emacs timer to retry at, say, one second intervals until success. That meant some extra complexity. More basically, it meant more delay until a command request and response could occur.</p>
<h1 id="flip">Flip</h1>
<p>So I decided to flip that around. Now, the stdin/stdout of the back end process is the command request/response channel. The same format of sexpr command requests and responses are sent — they just happen to go over ports that are stdin/stdout instead of TCP ports. The TCP server is for creating a REPL session, if/when that is desired. Make a TCP connection, read a session ID, et voila you have a new REPL session. The back end <code>parameterize</code>s the REPL’s <code>current-{input
output error}-mode</code> to the TCP ports. And fortunately, <code>comint-mode</code> makes it as easy to connect the buffer to I/O from a TCP connection, as to stdin/stdout of a process.</p>
<p>The other thing is, a process can have only one stdin/stdout. But of course it can accept multiple TCP connections. Now that we use the latter for REPL I/O, this removes a basic barrier to an old feature request: Supporting multiple REPLs would now be possible.</p>
<p>Of course “possible” is different than “implemented and working correctly”. So at first I focused mainly on the back end: Make sure that it wasn’t using global variables when it could use per-thread parameters, for things related to the REPL. Add a lookup table from unique REPL session IDs to information about each REPL session. Have the front end obtain the session ID and supply it later with commands — but otherwise, for now, the front end still just created one REPL session at a time.</p>
<h1 id="multiple-repls">Multiple REPLs</h1>
<p>After some time to get that working, I wanted to tackle supporting multiple REPLs in the front end. For a day or two, I procrastinated. I <em>really</em> did not want to write a bunch of “ceremonial” code related to REPL sessions. I did not want some whole UI for that, complete with more code and bugs.</p>
<p>I’m glad I hesitated, because I realized something simple. Emacs already has the concept that variable values can be global to all buffers — but also can be set so that a buffer has its own, local value.</p>
<p>So, assume a variable <code>racket-repl-buffer-name</code>. It only has meaning for a <code>racket-mode</code> buffer. It means, “what is the name of the <code>racket-repl-mode</code> buffer that commands like <code>racket-run</code> should use”. Initially such a variable could have a global value, <code>*Racket REPL*</code>. That default is the status quo behavior: All edit buffers take turns sharing the same, one and only REPL buffer.</p>
<p>Now, what if a <code>racket-mode</code> buffer for <code>foo.rkt</code> did a <code>(setq-local
racket-repl-buffer-name "*Racket REPL foo.rkt*")</code>? Now commands issued with that buffer selected would use (creating if necessary) a REPL buffer of <em>that</em> name. If every edit buffer did this, you’d have one REPL per edit buffer (much like DrRacket).</p>
<p>You could also imagine wanting edit buffers for files that are in the same <code>projectile</code> project, to share a REPL buffer: “One REPL per project”. OK: <code>setq-local</code> their <code>racket-repl-buffer-name</code> to some common name that includes the project name or root directory.</p>
<p>And so that is the mechanism I went with. A customization variable <code>racket-repl-buffer-name-function</code> points to a function that will set the <code>racket-repl-buffer-name</code> variable value for a <code>racket-mode</code> edit buffer when it is created. I defined three such functions, for the three strategies described above. And best of all, as a user you can supply your own such function.</p>
<p>I also added a <code>kill-buffer-hook</code>: When you kill a <code>racket-mode</code> edit buffer, and it is the last one using a <code>racket-repl-mode</code> buffer, we <code>y-or-n-prompt</code> offering to kill the REPL buffer, too. I figure that will help keep things cleaner, especially for the case where people want 1:1 edit:REPL buffers.</p>
<p>As a result, there is no Big UI about “sessions”. There are just edit buffers, and names of the REPL buffer(s) each should use. Because it rides directly on Emacs semantics for variables and buffers, there is non-trivial code that I did not need to write — and “out of sync” bugs that I could not create.</p>
<h1 id="that-cleaving-feeling">That cleaving feeling</h1>
<p>Of course there are <em>some</em> bugs lurking, about which I don’t yet know. Someday I’ll probably feel discouraged about this code, for reasons I don’t yet know. A lot of programming “innovation” amounts to trading an old set of problems for a new set. Sometimes the new problems aren’t better, they just feel better, at first, because novelty.</p>
<p>All stipulated and admitted. Even so, I <em>also</em> feel this was one of those situations where, finally, I saw a way to define a problem so that it aligns with the material I had to work with: Just a few gentle taps, and the material cleaves along the lines I need and want. That is a pretty awesome feeling. While it lasts.</p>
<h1 id="availability">Availability</h1>
<p>The stuff I’m describing in this blog post initially lived on a <code>new-comm</code> branch. I did just merge it back to the <code>check-syntax</code> branch. It’s <em>not</em> yet merged to <code>master</code>. I’m still reviewing documentation, dogfooding, and such. I’m coming up on two full months working on this, so, I’d like to merge to <code>master</code> in the near future.</p>
<footer></footer></article></div>Using check-syntax in Racket Modeurn:www-greghendershott-com:-2020-02-using-drracket-check-syntax-in-racket-mode.html2020-02-14T00:00:00Z2020-02-14T00:00:00ZGreg Hendershott
<div>
<article>
<header>
<h1>Using check-syntax in Racket Mode</h1>
<p class="date-and-tags">
<time datetime="2020-02-14" pubdate="true">2020-02-14</time> :: <span class="tags"><a href="/tags/Racket.html">Racket</a>, <a href="/tags/Emacs.html">Emacs</a></span></p></header>
<p>During most of January and into February, I’ve been working full-time to have <a href="https://github.com/greghendershott/racket-mode/">Racket Mode</a> make better use of <a href="https://docs.racket-lang.org/drracket-tools/Accessing_Check_Syntax_Programmatically.html">drracket/check-syntax</a> analysis of fully-expanded code. I’m pretty excited by it.</p><!-- more-->
<p>Consider this little Racket program:</p>
<div class="brush: racket">
<table class="sourcetable">
<tbody>
<tr>
<td class="linenos">
<div class="linenodiv">
<pre> 1
2
3
4
5
6
7
8
9
10</pre></div></td>
<td class="code">
<div class="source">
<pre><span></span><span class="kn">#lang </span><span class="nn">racket/base</span>
<span class="p">(</span><span class="n">push-the-red-button!</span><span class="p">)</span>
<span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/define.html#(form._((lib._racket/private/base..rkt)._define))" style="color: inherit">define</a></span> <span class="p">(</span><span class="n">f</span> <span class="n">x</span><span class="p">)</span>
<span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/generic-numbers.html#(def._((quote._~23~25kernel)._+))" style="color: inherit">+</a></span> <span class="n">x</span> <span class="mi">1</span><span class="p">))</span>
<span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/module.html#(form._((quote._~23~25kernel)._module))" style="color: inherit">module</a></span> <span class="n">m</span> <span class="n">typed/racket/base</span>
<span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/define.html#(form._((lib._racket/private/base..rkt)._define))" style="color: inherit">define</a></span> <span class="p">(</span><span class="n">g</span> <span class="n">x</span><span class="p">)</span>
<span class="nb"><a href="http://docs.racket-lang.org/reference/generic-numbers.html#(def._((quote._~23~25kernel)._+))" style="color: inherit">+</a></span> <span class="n">x</span> <span class="mi">1</span><span class="p">))</span>
</pre></div>
</td></tr></tbody></table>
</div>
<p>A few things to note:</p>
<ul>
<li>
<p>The file module has a side-effecting expression <code>(push-the-red-button!)</code>. Every time the file is evaluated a.k.a. “run”, this expression is evaluated.</p></li>
<li>
<p>The file module language is <code>racket/base</code>.</p></li>
<li>
<p>The submodule <code>m</code> language is <code>typed/racket/base</code>.</p></li></ul>
<h1 id="status-quo">Status quo</h1>
<p>Traditionally Racket Mode has required you to <code>racket-run</code> a file in order for some handy features to be available, because it can use the namespace resulting from <code>module->namespace</code> to support those features. This reflects its origins as a wrapper around <code>xrepl</code>. It retained that basic approach even as it grew beyond <code>xrepl</code>.</p>
<p>For instance it uses <code>namespace-mapped-symbols</code> to obtain a list of completion candidates. If it needs to get you documentation, it can synthesize a suitable identifier from a string using <code>namespace-symbol->identifier</code>. It can give such an identifier to <code>xref-binding->definition-tag</code>, and in turn give that result to <code>xref-tag->path+anchor</code>, and get the appropriate help topic.</p>
<p>Likewise the visit-definition feature can use the namespace to make the correct identifier to give to <code>identifier-binding</code>, which tells you where the binding was defined.</p>
<p>Racket Mode lets you run, not just the outermost, file’s module, but specific submodules — the innermost module around point. So when you run the <code>m</code> module, the namespace is inside that submodule. Therefore features work as you’d expect. For example help will be for Typed Racket’s <code>define</code>, and visit-definition will go to the source for that <code>define</code>.</p>
<p>This all works fine, with one drawback: You must <code>racket-run</code> the edit buffer to get results. (And if you make changes, you must run it again to get updated results.) This is a speed bump, especially when exploring many Racket files. It would be nice to open the file, and just visit definitions or get help — without needing to run each file. That can be slow. Worse, what about side-effects like <code>push-the-red-button!</code>?</p>
<p>Also, I knew at least a few people who use Racket Mode pretty much solely as an editing mode. In other words, they use <code>racket-mode</code> to edit .rkt files, but they rarely or never use <code>racket-run</code> and <code>racket-repl-mode</code>. Maybe <code>racket-mode</code> should be simpler and more lightweight — maybe some new minor mode should augment it with the “extras” for those who want them. This is a typical Emacs approach: A major mode handles a basic set of functionality, and one or more minor modes optionally enhance it.</p>
<h1 id="check-syntax">Check Syntax</h1>
<p>The Dr Racket IDE has a “Check Syntax” feature, which works solely by expanding a module — but <em>not</em> evaluating a.k.a. “running” it — and analyzing that fully-expanded syntax.</p>
<p>The acronym REPL — read, eval, print, loop? The “E” actually covers a few steps: expand, compile, evaluate. More like “RECEPL”.</p>
<p>A program in fully-expanded form looks like Racket “assembly language”. It is tedious to read, as a human. But a few key forms like <code>define-values</code>, <code>define-syntaxes</code>, and <code>#%require</code>, provide a lot of very useful information about bindings — things defined in that module or imported from another.<sup><a href="#using-drracket-check-syntax-in-racket-mode-footnote-1-definition" name="using-drracket-check-syntax-in-racket-mode-footnote-1-return">1</a></sup></p>
<p>The great news is that the <a href="https://docs.racket-lang.org/drracket-tools/Accessing_Check_Syntax_Programmatically.html">drracket/check-syntax</a> library exposes the analysis Dr Racket does to create annotations for the source buffer. I was able to leverage this to get most of what I wanted to do.</p>
<p>In a few cases, I need to extend this. For example, I wanted <em>all</em> local, module, and imported definitions to be available as completion candidates. The library produces annotations for the arrows Dr Racket draws between definitions and uses. If something is defined but not used? You can’t draw an arrow from something to nothing. Therefore it doesn’t produce an annotation. So the set of definitions in arrow annotations isn’t sufficient. Fortunately there is also a “mouseover text” annotation such as “N bound occurrence(s)”.<sup><a href="#using-drracket-check-syntax-in-racket-mode-footnote-2-definition" name="using-drracket-check-syntax-in-racket-mode-footnote-2-return">2</a></sup> So, a way to get all module and local definitions is to look for all such annotations — those are your definitions. As for imported definitions, I needed to walk the fully-expanded syntax myself, looking for <code>#%require</code> and its subforms that filter and rename what is imported, and do such filters/renames on <code>module->exports</code> lists. Although that didn’t take five minutes to figure out and debug, it wasn’t too bad.</p>
<p>Some of the work involved other issues. By far the slowest part isn’t the analysis of expanded code, it is getting that expanded code: <code>expand</code> can be non-trivial for languages like Typed Racket. For other reasons Racket Mode already maintained a cache of <code>file -> expanded syntax</code>. I needed that to also handle <code>(path-string code-string) -> expanded syntax</code>, for the case of unsaved buffer text. And as a result, I needed to make my own, cache-aware wrapper like <code>show-content</code> around <code>make-traversal</code>.</p>
<p>I also enhanced the back end command server to support the idea of cancelling a command already in progress. That way when we get a new request to analyze updated source, we can cancel any analysis possibly still running for the old, “stale” text.</p>
<h1 id="front-end">Front end</h1>
<p>How about the “front end”, implemented in Emacs Lisp?</p>
<p>Racket Mode already had a <code>racket-check-syntax-mode</code> minor mode. However it made the buffer read-only. You could navigate, and rename bindings, but you had to quit the mode to resume normal free-form editing. Also it didn’t attempt to unify the UX with normal commands to visit definitions, see help, and so on.</p>
<p>Instead, I wanted to this be something that — as in Dr Racket — runs automatically in the background on a read/write buffer. After you make some changes, and after some short idle delay, a new analysis kicks off. When its results are ready, the buffer annotations and completion candidates are refreshed. Annotations are done using Emacs text properties, and in the case of defs and uses which refer to each other, use Emacs <em>markers</em> so they remain valid after most edits. When there are syntax errors, those are annotated, but previous completion candidates are retained.</p>
<p>Although we can’t draw graphical arrows, we can highlight definitions and uses when point moves over them, using <code>cursor-sensor-mode</code> (which is a minor mode a user can disable if they find this slow or distracting).</p>
<p>Initially I still called this <code>racket-check-syntax-mode</code>. But (a) that’s pretty verbose, and (b) checking your syntax for mistakes is not most of the benefit. Really, it is a static analysis of expanded code that helps you explore and explain. With all those “exp” prefixes, plus an inclination for an even shorter name, I settled on <code>racket-xp-mode</code>.</p>
<p>At this point I’ll simply quote the documentation string for <code>racket-xp-mode</code> and show some screen shots:</p>
<blockquote>
<p>racket-xp-mode is an interactive function defined in racket-xp.el.</p>
<p><strong>Documentation</strong></p>
<p>A minor mode that analyzes expanded code to explain and explore.</p>
<p>This minor mode is an optional enhancement to racket-mode edit buffers. Like any minor mode, you can turn it on or off for a specific buffer. If you always want to use it, put the following code in your Emacs init file:</p>
<div class="brush: elisp">
<table class="sourcetable">
<tbody>
<tr>
<td class="linenos">
<div class="linenodiv">
<pre>1
2</pre></div></td>
<td class="code">
<div class="source">
<pre><span></span><span class="p">(</span><span class="nb">require</span> <span class="ss">'racket-xp</span><span class="p">)</span>
<span class="p">(</span><span class="nv">add-hook</span> <span class="ss">'racket-mode-hook</span> <span class="nf">#'</span><span class="nv">racket-xp-mode</span><span class="p">)</span>
</pre></div>
</td></tr></tbody></table>
</div>
<p>Note: This mode won’t do anything unless/until the Racket Mode back end is running. It will try to start the back end automatically. You do <em>not</em> need to racket-run the buffer you are editing.</p>
<p>This mode uses the drracket/check-syntax package to analyze fully-expanded programs, without needing to evaluate a.k.a. “run” them. The resulting analysis provides information for:</p>
<ul>
<li>
<p>Visually annotating bindings — local or imported definitions and references to them.</p></li>
<li>
<p>Completion candidates.</p></li>
<li>
<p>Defintions’ source and documentation.</p></li></ul>
<p>When point is on a definition or use, related items are highlighted using racket-xp-def-face and racket-xp-use-face — instead of drawing arrows as in Dr Racket — and “mouse over”. Information is displayed using the function(s) in the hook variable racket-show-functions; it is also available when hovering the mouse cursor. Note: If you find these features too distracting and/or slow, you may disable cursor-sensor-mode. The remaining features discussed below will still work.</p>
<p>You may also use commands to navigate among a definition and its uses, or to rename a local definitions and all its uses.</p>
<p>In the following little example, not only does drracket/check-syntax distinguish the various “x” bindings, it understands the two different imports of “define”:</p>
<div class="brush: racket">
<table class="sourcetable">
<tbody>
<tr>
<td class="linenos">
<div class="linenodiv">
<pre>1
2
3
4
5
6
7
8</pre></div></td>
<td class="code">
<div class="source">
<pre><span></span><span class="kn">#lang </span><span class="nn">racket/base</span>
<span class="p">(</span><span class="k">define</span> <span class="n">x</span> <span class="mi">1</span><span class="p">)</span>
<span class="n">x</span>
<span class="p">(</span><span class="k">let</span> <span class="p">([</span><span class="n">x</span> <span class="n">x</span><span class="p">])</span>
<span class="p">(</span><span class="nb">+</span> <span class="n">x</span> <span class="mi">1</span><span class="p">))</span>
<span class="p">(</span><span class="k">module</span> <span class="n">m</span> <span class="n">typed/racket/base</span>
<span class="p">(</span><span class="k">define</span> <span class="n">x</span> <span class="mi">2</span><span class="p">)</span>
<span class="n">x</span><span class="p">)</span>
</pre></div>
</td></tr></tbody></table>
</div>
<p>The function racket-xp-complete-at-point is added to the variable completion-at-point-functions. Note that in this case, it is not smart about submodules; identifiers are assumed to be definitions from the file’s module or its imports. In addition to supplying completion candidates, it supports the ":company-location" property to inspect the definition of a candidate and the ":company-doc-buffer" property to view its documentation.</p>
<p>When you edit the buffer, existing annotations are retained; their positions are updated to reflect the edit. Annotations for new or deleted text are not requested until after racket-xp-after-change-refresh-delay seconds. The request is made asynchronously so that Emacs will not block — for moderately complex source files, it can take some seconds simply to fully expand them, as well as a little more time for the drracket/check-syntax analysis. When the results are ready, all annotations for the buffer are completely refreshed.</p>
<p>You may also set racket-xp-after-change-refresh-delay to nil and use the racket-xp-annotate command manually.</p>
<p>The mode line changes to reflect the current status of annotations, and whether or not you had a syntax error.</p>
<p>If you have one or more syntax errors, use the standard next-error command and key bindings to navigate among them. Although most languages will stop after the first syntax error, some like Typed Racket will try to collect and report multiple errors.</p>
<p>Tip: This mode follows the convention that a minor mode may only use a prefix key consisting of “C-c” followed by a punctuation key. As a result, racket-xp-control-c-hash-keymap is bound to “C-c #” by default. Although you might find this awkward to type, remember that as an Emacs user, you are free to bind this map to a more convenient prefix, and/or bind any individual commands directly to whatever keys you prefer.</p>
<pre><code>C-c # . racket-xp-visit-definition
C-c # g racket-xp-annotate
C-c # j racket-xp-next-definition
C-c # k racket-xp-previous-definition
C-c # n racket-xp-next-use
C-c # p racket-xp-previous-use
C-c # r racket-xp-rename
C-c C-. racket-xp-describe
C-c C-d racket-xp-documentation
M-. racket-xp-visit-definition</code></pre></blockquote>
<figure><img src="/img/check-syntax.gif" width="100%" />
<figcaption>Animatation of check-syntax screen shots</figcaption></figure>
<p>Keep in mind that almost everything can be customized using defined faces or variables. Information like “1 bound occurrence” or error messages can be configured to be shown using the echo area (as in these screen shots), a header line, a <code>pos-tip</code> tooltip, and/or whatever custom function you want to supply.<sup><a href="#using-drracket-check-syntax-in-racket-mode-footnote-3-definition" name="using-drracket-check-syntax-in-racket-mode-footnote-3-return">3</a></sup></p>
<p>It was particularly fun when things were working well enough that I could use <code>racket-xp-mode</code> to more quickly explore and better understand some of the <a href="https://docs.racket-lang.org/drracket-tools/Accessing_Check_Syntax_Programmatically.html">drracket/check-syntax</a> code — as well as my own.</p>
<h1 id="taking-stock">Taking stock</h1>
<p>What remained was to step back and look at the <code>racket-mode</code> major mode (in hindsight maybe better-named <code>racket-edit-mode</code>), now optionally enhanced by <code>racket-xp-mode</code>, as well as the <code>racket-repl-mode</code> major mode.</p>
<p>What should live where, what should change, and what should remain the same?</p>
<p>After all, the old way of using the namespace from a run program, is actually still useful and quite appropriate for the REPL. You can run a program, then go into the REPL, and <code>define</code> and <code>require</code> all sorts of things. It still makes send to have that live namespace answer questions like “What are your symbols for completion candiates?” or “Which <code>define</code> is this, exactly, and where is its help or definition source?”. It is nice that you no longer <em>must</em> run your program, but if you <em>have</em> run it, and you’re in the REPL buffer… when in Rome.</p>
<p>So I realized there were a few commands — visit-definition, describe, help, completions — where there should be two flavors. One provided by <code>racket-xp-mode</code> and another provided by <code>racket-repl-mode</code> still working the “old” way. For example the old <code>racket-visit-definition</code> is effectively just moved/renamed to <code>racket-repl-visit-definition</code>.</p>
<p>At a lower level, also, there were some functions where I wrapped both flavors in a single back end command with a “how” argument: Use the <code>current-namespace</code> from <code>module->namespace</code>, or, use the namespace and module lexical context for some given path? (The latter covers cases where there exists no annotation, or the user has typed some arbitrary text, and we want to synthesize an identifier.)</p>
<h1 id="automatically-starting-the-back-end">Automatically starting the back end</h1>
<p>One other note. Racket Mode consists of an Emacs front end, obviously, and also a back end command server written in Racket. Although <code>racket-xp-mode</code> does <em>not</em> need to <code>racket-run</code> any .rkt buffer, it <em>does</em> need to talk to the back end server. Today that still means that a <code>racket-repl-mode</code> buffer is created. It just stays in the background, and it needn’t be running any particular file — but it must be open.<sup><a href="#using-drracket-check-syntax-in-racket-mode-footnote-4-definition" name="using-drracket-check-syntax-in-racket-mode-footnote-4-return">4</a></sup></p>
<p>When <code>racket-xp-mode</code> needs the back end, it will try to start it if not already running. This made me nervous, because auto-starting in the past led to some bugs, especially when something needed a command response to continue: Emacs could sometimes get stuck at a “starting…” or “waiting for…” message. You could
<kbd>C-g</kbd> out of that, usually. But it sucked. I had a think and realized the key was to make everything “asynchronous” with explicit continuations. I’d already replaced many uses of <code>racket--cmd/await</code> with <code>racket--cmd/async</code> — which takes a completion callback a.k.a. continuation procedure. I replaced even more. And also, I realized that starting the command server should itself be asynchronous. And so any <code>racket--cmd/async</code> that can’t happen yet, is pushed onto a list of continuations to be run eventually by the “connected to command server” continuation when it is called. Meanwhile, Emacs remains unblocked. Although I am not going to jinx this by declaring “mission accomplished!”, and some bugs may lurk, I’ve tried to be diligent in thinking through race conditions. I feel like this approach at least simplifies what needs to be considered.</p>
<h1 id="availability">Availability</h1>
<p>As I write this, I have 100+ commits still on a <code>check-syntax</code> branch — not yet merged to <code>master</code> or available from MELPA.</p>
<p>If you’re feeling brave or curious, feel free to try it out! I’ve been dogfooding it a fair amount. I’m still finding and fixing edge cases, but I think it’s stable enough to understand the basic experience.</p>
<div class="footnotes">
<ol>
<li id="using-drracket-check-syntax-in-racket-mode-footnote-1-definition" class="footnote-definition">
<p>The outermost <code>module</code> form also gets a syntax-property with the lexical context inside the module, including imports. As a result, you can synthesize an identifer using that as the first argument to <code>datum->syntax</code>, and then do the same things with that identifier you could with one from <code>namespace-symbol->identifer</code>, in order to support features like visit-definition or help, where the user has supplied some arbitrary string. <a href="#using-drracket-check-syntax-in-racket-mode-footnote-1-return">↩</a></p></li>
<li id="using-drracket-check-syntax-in-racket-mode-footnote-2-definition" class="footnote-definition">
<p>And if you want to find <em>unused</em> imported and locally-defined definitions, they’ll have a “mouseover” annotation, “no bound occurrences”. <a href="#using-drracket-check-syntax-in-racket-mode-footnote-2-return">↩</a></p></li>
<li id="using-drracket-check-syntax-in-racket-mode-footnote-3-definition" class="footnote-definition">
<p>It’s also shown when you hover the mouse pointer. If you use a mouse with Emacs, I won’t judge you. <em>*coughs*</em>. <a href="#using-drracket-check-syntax-in-racket-mode-footnote-3-return">↩</a></p></li>
<li id="using-drracket-check-syntax-in-racket-mode-footnote-4-definition" class="footnote-definition">
<p>Perhaps in the future that will change — especially if Racket Mode ever supports multiple REPLs open at once. In that case, starting the back end will be an independent first step, things like <code>racket-run</code> will create one or more REPLs, and the I/O for the comint buffers will be a network stream instead of stdin/stdout. <a href="#using-drracket-check-syntax-in-racket-mode-footnote-4-return">↩</a></p></li></ol></div>
<footer></footer></article></div>Future of Racketurn:www-greghendershott-com:-2019-07-future-of-racket.html2019-07-28T00:00:00Z2019-07-28T00:00:00ZGreg Hendershott
<div>
<article>
<header>
<h1>Future of Racket</h1>
<p class="date-and-tags">
<time datetime="2019-07-28" pubdate="true">2019-07-28</time> :: <span class="tags"><a href="/tags/Racket.html">Racket</a></span></p></header>
<p>For most of the last decade I’ve <a href="https://www.greghendershott.com/2014/09/written-in-racket.html">made things in Racket</a> — including making <a href="https://www.racket-mode.com/">tools</a> and <a href="https://www.greghendershott.com/fear-of-macros/">tutorials</a> to support other people making things in Racket.</p>
<p>At RacketCon 2019, Aaron Turon gave the keynote about the Rust community.</p>
<p>That afternoon, I had a talk about Racket Mode for Emacs.</p>
<p>The next morning?</p><!-- more-->
<p>Matthew Flatt gave his State of Racket talk. He said the community is growing well. Next, he proposed, let’s change the surface syntax away from lispy s-expressions, to something that will be a lower barrier of entry to new users.</p>
<p>He said it’s just a proposal. It would probably take a couple years. <code>#lang racket</code> programs would continue to work meanwhile and after the change. But eventually, in his proposal, the Racket culture should use this new syntax. (“Culture” meaning documentation, discussions, books, tutorials, tools.)</p>
<p>To be fair, he gave people a lot of time to share their initial reactions and answer their questions.</p>
<p>Also to be fair, if you have some mileage on your odometer, your pattern-recognition machinery will quickly assign high probabilities to various outcomes.</p>
<p>I’m concerned the change won’t help grow the community; instead hurt it. I’ve explained why in a few racket-users posts.<sup><a href="#future-of-racket-footnote-1-definition" name="future-of-racket-footnote-1-return">1</a></sup> I won’t here. I don’t think it matters. All you can say is, “Hey, that stove is hot, you might not want to touch it.”</p>
<p>I don’t regret contributing things and trying to help the community grow. I learned a lot! I had fun! On the other hand, if I’ll no longer learn or have fun? I’m not getting any younger. There are other things to do.</p>
<p>In the near future, I’m spending more time with other programming languages (currently Rust and Haskell). Possibly less time programming, at all. We’ll see.</p>
<p>Who cares? Why write this post?</p>
<ul>
<li>
<p>To have a background explanation I can simply link to from things like README files, as needed from time to time.</p></li>
<li>
<p>Emotionally, I can’t quite bring myself to say, “So Long and Thanks for All the Standard-Fish!” That feels too abrupt. But it’s probably realistic about my current level of mental separation. So if there are a few people making plans, in part based on my own plans, I want to be up-front.</p></li></ul>
<p>Finally, I sincerely wish all the best for the Racket core team and the community, which in my experience consists of amazing people with good intentions, open hearts, and sharp brains.</p>
<div class="footnotes">
<ol>
<li id="future-of-racket-footnote-1-definition" class="footnote-definition">
<p>Update 2019–08–10: Now that some people are linking to this blog post? Here are links to my racket-users posts: <a href="https://groups.google.com/d/msg/racket-users/ewWuCvbe93k/s_P94KBmCAAJ">blah</a>, <a href="https://groups.google.com/d/msg/racket-users/ewWuCvbe93k/dEHdpFZnCAAJ">blah</a>, <a href="https://groups.google.com/d/msg/racket-users/ewWuCvbe93k/aiskVQd1CAAJ">blah</a>, <a href="https://groups.google.com/d/msg/racket-users/ewWuCvbe93k/WpIl8bsWCgAJ">blah</a>, and <a href="https://groups.google.com/d/msg/racket-users/vN_1uUJZnXo/5bXiMEBvCgAJ">blah</a>. Also I updated this to “linkify” some other things and add Haskell to the list of languages I’m mostly focusing on now. <a href="#future-of-racket-footnote-1-return">↩</a></p></li></ol></div>
<footer></footer></article></div>Supporting multi-inurn:www-greghendershott-com:-2019-04-supporting-multi-in.html2019-04-25T00:00:00Z2019-04-25T00:00:00ZGreg Hendershott
<div>
<article>
<header>
<h1>Supporting multi-in</h1>
<p class="date-and-tags">
<time datetime="2019-04-25" pubdate="true">2019-04-25</time> :: <span class="tags"><a href="/tags/Racket.html">Racket</a>, <a href="/tags/Emacs.html">Emacs</a></span></p></header>
<p>In <a href="https://github.com/greghendershott/racket-mode/">racket-mode</a> I improved support for the <code><a href="http://docs.racket-lang.org/reference/require.html#(form._((lib._racket/require..rkt)._multi-in))" style="color: inherit">multi-in</a></code> form provided by <code>racket/require</code>.</p><!-- more-->
<h1 id="what-is-brush-racketmulti-in">What is <code><a href="http://docs.racket-lang.org/reference/require.html#(form._((lib._racket/require..rkt)._multi-in))" style="color: inherit">multi-in</a></code>?</h1>
<p>Instead of:</p>
<div class="brush: racket">
<table class="sourcetable">
<tbody>
<tr>
<td class="linenos">
<div class="linenodiv">
<pre>1
2
3
4
5</pre></div></td>
<td class="code">
<div class="source">
<pre><span></span><span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/require.html#(form._((lib._racket/private/base..rkt)._require))" style="color: inherit">require</a></span> <span class="n">net/uri-codec</span>
<span class="n">net/url</span>
<span class="n">racket/contract</span>
<span class="n">racket/format</span>
<span class="n">racket/string</span><span class="p">)</span>
</pre></div>
</td></tr></tbody></table>
</div>
<p>You can say:</p>
<div class="brush: racket">
<table class="sourcetable">
<tbody>
<tr>
<td class="linenos">
<div class="linenodiv">
<pre>1
2
3</pre></div></td>
<td class="code">
<div class="source">
<pre><span></span><span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/require.html#(form._((lib._racket/private/base..rkt)._require))" style="color: inherit">require</a></span> <span class="n">racket/require</span>
<span class="p">(</span><span class="n"><a href="http://docs.racket-lang.org/reference/require.html#(form._((lib._racket/require..rkt)._multi-in))" style="color: inherit">multi-in</a></span> <span class="n">net</span> <span class="p">(</span><span class="n">uri-codec</span> <span class="n">url</span><span class="p">))</span>
<span class="p">(</span><span class="n"><a href="http://docs.racket-lang.org/reference/require.html#(form._((lib._racket/require..rkt)._multi-in))" style="color: inherit">multi-in</a></span> <span class="n">racket</span> <span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/attaching-contracts-to-values.html#(form._((lib._racket/contract/private/base..rkt)._contract))" style="color: inherit">contract</a></span> <span class="nb"><a href="http://docs.racket-lang.org/reference/Writing.html#(def._((quote._~23~25kernel)._format))" style="color: inherit">format</a></span> <span class="nb"><a href="http://docs.racket-lang.org/reference/strings.html#(def._((quote._~23~25kernel)._string))" style="color: inherit">string</a></span><span class="p">)))</span>
</pre></div>
</td></tr></tbody></table>
</div>
<p>One detail: The <code>racket/require</code> must appear before any <code>multi-in</code> forms. Any sorting must make an exception for this.</p>
<h1 id="what-are-the-racket-tidy-trim-base-requires-commands">What are the <code>racket-{tidy trim base}-requires</code> commands?</h1>
<ul>
<li>
<p><code>racket-tidy-requires</code> gathers multiple <code>require</code> forms into one. Within that, it groups phase level sub-forms such as <code>for-syntax</code>. Finally it sorts absolute module paths like <code>foo</code> before relative paths like <code>"foo.rkt"</code>, and sorts each of those alphabetically.</p></li>
<li>
<p><code>racket-trim-requires</code> uses the <code>show-requires</code> function from <code>macro-debugger/analysis/check-requires</code> to analyze requires and delete any unused. Also it tidies.</p></li>
<li>
<p><code>racket-base-requires</code> uses the analysis to change a <code>#lang racket</code> file to <code>#lang racket/base</code>, adding requires as necessary. Also it trims and tidies.</p></li></ul>
<h1 id="what-are-the-changes">What are the changes?</h1>
<ol>
<li>
<p>Sorting makes sure to keep <code>racket/require</code> first.</p></li>
<li>
<p>If <code>racket/require</code> is present in the original <code>require</code> form(s), then <code>multi-in</code> is used when tidying.</p></li>
<li>
<p>Any analysis that says requires should be removed, should handle <code>multi-in</code> forms.</p></li></ol>
<p>Essentially, the commands first “expand” or “explode” <code>multi-in</code> forms to individual requires, do any analysis and modifications, then try to “unexpand” or “implode” them back again.<sup><a href="#supporting-multi-in-footnote-1-definition" name="supporting-multi-in-footnote-1-return">1</a></sup></p>
<p>All together, these changes close issues <a href="https://github.com/greghendershott/racket-mode/issues/355">355</a>, <a href="https://github.com/greghendershott/racket-mode/issues/356">356</a>, and <a href="https://github.com/greghendershott/racket-mode/issues/369">369</a>.</p>
<h2 id="example-without-racketrequire">Example without <code>racket/require</code></h2>
<p>Here’s an example where <code>racket/require</code> is <em>not</em> in the original <code>require</code> forms:</p>
<div class="brush: racket">
<table class="sourcetable">
<tbody>
<tr>
<td class="linenos">
<div class="linenodiv">
<pre>1
2
3
4
5
6
7</pre></div></td>
<td class="code">
<div class="source">
<pre><span></span><span class="kn">#lang </span><span class="nn">racket</span>
<span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/require.html#(form._((lib._racket/private/base..rkt)._require))" style="color: inherit">require</a></span> <span class="n">net/url</span><span class="p">)</span>
<span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/require.html#(form._((lib._racket/private/base..rkt)._require))" style="color: inherit">require</a></span> <span class="n">net/uri-codec</span><span class="p">)</span>
<span class="c1">;; Just some expressions using imported definitions</span>
<span class="nb"><a href="http://docs.racket-lang.org/reference/strings.html#(def._((lib._racket/string..rkt)._string-join))" style="color: inherit">string-join</a></span> <span class="nb"><a href="http://docs.racket-lang.org/reference/strings.html#(def._((lib._racket/format..rkt)._~7ea))" style="color: inherit">~a</a></span> <span class="n"><a href="http://docs.racket-lang.org/net/url.html#(def._((lib._net/url..rkt)._get-pure-port))" style="color: inherit">get-pure-port</a></span> <span class="n"><a href="http://docs.racket-lang.org/net/uri-codec.html#(def._((lib._net/uri-codec..rkt)._uri-decode))" style="color: inherit">uri-decode</a></span> <span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/match.html#(form._((lib._racket/match..rkt)._match-define))" style="color: inherit">match-define</a></span> <span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/pairs.html#(def._((quote._~23~25kernel)._list))" style="color: inherit">list</a></span> <span class="n">n</span><span class="p">)</span> <span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/pairs.html#(def._((quote._~23~25kernel)._list))" style="color: inherit">list</a></span> <span class="mi">1</span><span class="p">))</span>
</pre></div>
</td></tr></tbody></table>
</div>
<p>After
<kbd>M-x</kbd> <code>racket-base-requires</code>:</p>
<div class="brush: racket">
<table class="sourcetable">
<tbody>
<tr>
<td class="linenos">
<div class="linenodiv">
<pre> 1
2
3
4
5
6
7
8
9
10</pre></div></td>
<td class="code">
<div class="source">
<pre><span></span><span class="kn">#lang </span><span class="nn">racket/base</span>
<span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/require.html#(form._((lib._racket/private/base..rkt)._require))" style="color: inherit">require</a></span> <span class="n">net/uri-codec</span>
<span class="n">net/url</span>
<span class="n">racket/format</span>
<span class="n">racket/match</span>
<span class="n">racket/string</span><span class="p">)</span>
<span class="c1">;; Just some expressions using imported definitions</span>
<span class="nb"><a href="http://docs.racket-lang.org/reference/strings.html#(def._((lib._racket/string..rkt)._string-join))" style="color: inherit">string-join</a></span> <span class="nb"><a href="http://docs.racket-lang.org/reference/strings.html#(def._((lib._racket/format..rkt)._~7ea))" style="color: inherit">~a</a></span> <span class="n"><a href="http://docs.racket-lang.org/net/url.html#(def._((lib._net/url..rkt)._get-pure-port))" style="color: inherit">get-pure-port</a></span> <span class="n"><a href="http://docs.racket-lang.org/net/uri-codec.html#(def._((lib._net/uri-codec..rkt)._uri-decode))" style="color: inherit">uri-decode</a></span> <span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/match.html#(form._((lib._racket/match..rkt)._match-define))" style="color: inherit">match-define</a></span> <span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/pairs.html#(def._((quote._~23~25kernel)._list))" style="color: inherit">list</a></span> <span class="n">n</span><span class="p">)</span> <span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/pairs.html#(def._((quote._~23~25kernel)._list))" style="color: inherit">list</a></span> <span class="mi">1</span><span class="p">))</span>
</pre></div>
</td></tr></tbody></table>
</div>
<h2 id="example-with-racketrequire">Example with <code>racket/require</code></h2>
<p>Here’s an example where <code>racket/require</code> <em>is</em> in the original <code>require</code> forms:</p>
<div class="brush: racket">
<table class="sourcetable">
<tbody>
<tr>
<td class="linenos">
<div class="linenodiv">
<pre>1
2
3
4
5
6
7
8</pre></div></td>
<td class="code">
<div class="source">
<pre><span></span><span class="kn">#lang </span><span class="nn">racket</span>
<span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/require.html#(form._((lib._racket/private/base..rkt)._require))" style="color: inherit">require</a></span> <span class="n">racket/require</span><span class="p">)</span> <span class="c1">;new</span>
<span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/require.html#(form._((lib._racket/private/base..rkt)._require))" style="color: inherit">require</a></span> <span class="n">net/url</span><span class="p">)</span>
<span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/require.html#(form._((lib._racket/private/base..rkt)._require))" style="color: inherit">require</a></span> <span class="n">net/uri-codec</span><span class="p">)</span>
<span class="c1">;; Just some expressions using imported definitions</span>
<span class="nb"><a href="http://docs.racket-lang.org/reference/strings.html#(def._((lib._racket/string..rkt)._string-join))" style="color: inherit">string-join</a></span> <span class="nb"><a href="http://docs.racket-lang.org/reference/strings.html#(def._((lib._racket/format..rkt)._~7ea))" style="color: inherit">~a</a></span> <span class="n"><a href="http://docs.racket-lang.org/net/url.html#(def._((lib._net/url..rkt)._get-pure-port))" style="color: inherit">get-pure-port</a></span> <span class="n"><a href="http://docs.racket-lang.org/net/uri-codec.html#(def._((lib._net/uri-codec..rkt)._uri-decode))" style="color: inherit">uri-decode</a></span> <span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/match.html#(form._((lib._racket/match..rkt)._match-define))" style="color: inherit">match-define</a></span> <span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/pairs.html#(def._((quote._~23~25kernel)._list))" style="color: inherit">list</a></span> <span class="n">n</span><span class="p">)</span> <span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/pairs.html#(def._((quote._~23~25kernel)._list))" style="color: inherit">list</a></span> <span class="mi">1</span><span class="p">))</span>
</pre></div>
</td></tr></tbody></table>
</div>
<p>After
<kbd>M-x</kbd> <code>racket-base-requires</code>:</p>
<div class="brush: racket">
<table class="sourcetable">
<tbody>
<tr>
<td class="linenos">
<div class="linenodiv">
<pre>1
2
3
4
5
6
7
8</pre></div></td>
<td class="code">
<div class="source">
<pre><span></span><span class="kn">#lang </span><span class="nn">racket/base</span>
<span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/require.html#(form._((lib._racket/private/base..rkt)._require))" style="color: inherit">require</a></span> <span class="n">racket/require</span>
<span class="p">(</span><span class="n"><a href="http://docs.racket-lang.org/reference/require.html#(form._((lib._racket/require..rkt)._multi-in))" style="color: inherit">multi-in</a></span> <span class="n">net</span> <span class="p">(</span><span class="n">uri-codec</span> <span class="n">url</span><span class="p">))</span>
<span class="p">(</span><span class="n"><a href="http://docs.racket-lang.org/reference/require.html#(form._((lib._racket/require..rkt)._multi-in))" style="color: inherit">multi-in</a></span> <span class="n">racket</span> <span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/Writing.html#(def._((quote._~23~25kernel)._format))" style="color: inherit">format</a></span> <span class="k"><a href="http://docs.racket-lang.org/reference/match.html#(form._((lib._racket/match..rkt)._match))" style="color: inherit">match</a></span> <span class="nb"><a href="http://docs.racket-lang.org/reference/strings.html#(def._((quote._~23~25kernel)._string))" style="color: inherit">string</a></span><span class="p">)))</span>
<span class="c1">;; Just some expressions using imported definitions</span>
<span class="nb"><a href="http://docs.racket-lang.org/reference/strings.html#(def._((lib._racket/string..rkt)._string-join))" style="color: inherit">string-join</a></span> <span class="nb"><a href="http://docs.racket-lang.org/reference/strings.html#(def._((lib._racket/format..rkt)._~7ea))" style="color: inherit">~a</a></span> <span class="n"><a href="http://docs.racket-lang.org/net/url.html#(def._((lib._net/url..rkt)._get-pure-port))" style="color: inherit">get-pure-port</a></span> <span class="n"><a href="http://docs.racket-lang.org/net/uri-codec.html#(def._((lib._net/uri-codec..rkt)._uri-decode))" style="color: inherit">uri-decode</a></span> <span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/match.html#(form._((lib._racket/match..rkt)._match-define))" style="color: inherit">match-define</a></span> <span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/pairs.html#(def._((quote._~23~25kernel)._list))" style="color: inherit">list</a></span> <span class="n">n</span><span class="p">)</span> <span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/pairs.html#(def._((quote._~23~25kernel)._list))" style="color: inherit">list</a></span> <span class="mi">1</span><span class="p">))</span>
</pre></div>
</td></tr></tbody></table>
</div>
<div class="footnotes">
<ol>
<li id="supporting-multi-in-footnote-1-definition" class="footnote-definition">
<p>Not everything survives a round-trip, exactly. A Cartesian product like <code>(multi-in (a b) (c d))</code> will end up as a <code>(multi-in a (c d))</code> and a <code>(multi-in b (c d))</code> — equivalent but not as concise. <a href="#supporting-multi-in-footnote-1-return">↩</a></p></li></ol></div>
<footer></footer></article></div>Exploding Frogurn:www-greghendershott-com:-2019-04-exploding-frog.html2019-04-03T00:00:00Z2019-04-03T00:00:00ZGreg Hendershott
<div>
<article>
<header>
<h1>Exploding Frog</h1>
<p class="date-and-tags">
<time datetime="2019-04-03" pubdate="true">2019-04-03</time> :: <span class="tags"><a href="/tags/Racket.html">Racket</a>, <a href="/tags/Frog.html">Frog</a></span></p></header>
<p>I’m writing and publishing this post using something other than <a href="https://github.com/greghendershott/frog">Frog</a>.</p>
<p>Having said that, I’m not planning to abandon maintaining Frog.</p><!-- more-->
<p>Over the past week I explored doing something I’ve wanted to try for many years: “Exploding” Frog from an app that you run, into <a href="https://github.com/greghendershott/blog">a set of little commands that you call from a Makefile</a>.</p>
<p>I believe there’s a variation of <a href="https://en.wikipedia.org/wiki/Greenspun's_tenth_rule">Greenspun’s Tenth Rule</a>:</p>
<blockquote>
<p>Any sufficiently complicated static blog generator contains an ad-hoc, informally-specified, bug-ridden, slow implementation of half of <code>make</code>.</p>
<footer>-- Someone who has made a static blog generator</footer></blockquote>
<p>If you look at Frog’s code, by volume, an awful lot of it consists of:</p>
<ol>
<li>Figuring out what needs to be (re)built.</li>
<li>“Path math”.</li>
<li>Configuration, customization, templates.</li>
<li>Support for Bootstrap or various surveillance capitalism gadgets.</li></ol>
<p>On the one hand, it is certainly very “Rackety” to write this in Racket. On the other hand, Racket advocates language-oriented programming. Make is a language for expressing the first two things.</p>
<p>As for the third item: Originally, I thought it would be cool to write Frog as an app, thinking people might use it even if they weren’t particularly into Racket programming. In reality, of course approximately 103.8% of Frog users like to program in Racket. So maybe, call this crazy, programmers who want a blog generating program to work a little differently could, I don’t know, change the program a little. Maybe there need not be layers of configuration and customization.</p>
<hr />
<p>And yet, I’ve never really known that much about GNU Make. Just enough to cargo cult. And the GNU Make documentation, although thorough, never really “clicked” for me. This time, though, I stuck with it. It helped to have a non-trivial project in mind.</p>
<p>The most confusing thing for me at first, was, Make seems oriented around defining targets and “pulling” those from sources. “This executable depends on linking these object files. These object files depend on these .c and .h files.” I didn’t really grok how to make it be “push”-driven: We have a bunch of (say) <code>.md</code> post sources. Yes those get built into <code>.html</code> files, but we also have tags, and feeds, and…?</p>
<p>In the end I made use of some variables to define the sources, intermediate build files, and final outputs:</p>
<div class="brush: make">
<table class="sourcetable">
<tbody>
<tr>
<td class="linenos">
<div class="linenodiv">
<pre> 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21</pre></div></td>
<td class="code">
<div class="source">
<pre><span></span><span class="c"># Configure where are sources, build cache, and root of the web site</span>
<span class="c"># output.</span>
<span class="nv">src</span> <span class="o">:=</span> src
<span class="nv">cache</span> <span class="o">:=</span> .cache
<span class="nv">www</span> <span class="o">:=</span> www
<span class="c"># Make normally "pulls" targets from sources, but we want to "push"</span>
<span class="c"># sources to targets. As a result, we need to build lists of sources</span>
<span class="c"># and from those build lists of targets.</span>
<span class="nv">post-sources</span> <span class="o">:=</span> <span class="k">$(</span>shell find <span class="k">$(</span>src<span class="k">)</span>/posts -type f<span class="k">)</span>
<span class="nv">post-caches</span> <span class="o">:=</span> <span class="k">$(</span>patsubst <span class="k">$(</span>src<span class="k">)</span>/posts/%.md,<span class="k">$(</span>cache<span class="k">)</span>/%.rktd,<span class="k">$(</span>post-sources<span class="k">))</span>
<span class="nv">post-htmls</span> <span class="o">:=</span> <span class="k">$(</span>patsubst <span class="k">$(</span>cache<span class="k">)</span>/%.rktd,<span class="k">$(</span>www<span class="k">)</span>/%.html,<span class="k">$(</span>post-caches<span class="k">))</span>
<span class="nv">tag-caches</span> <span class="o">:=</span> <span class="k">$(</span>wildcard <span class="k">$(</span>cache<span class="k">)</span>/tags/*<span class="k">)</span>
<span class="nv">tag-htmls</span> <span class="o">:=</span> <span class="k">$(</span>patsubst %,<span class="k">$(</span>www<span class="k">)</span>/tags/%.html,<span class="k">$(</span>notdir <span class="k">$(</span>tag-caches<span class="k">)))</span>
<span class="nv">tag-atom-feeds</span> <span class="o">:=</span> <span class="k">$(</span>patsubst %,<span class="k">$(</span>www<span class="k">)</span>/feeds/%.atom.xml,<span class="k">$(</span>notdir <span class="k">$(</span>tag-caches<span class="k">)))</span>
<span class="nv">tag-rss-feeds</span> <span class="o">:=</span> <span class="k">$(</span>patsubst %,<span class="k">$(</span>www<span class="k">)</span>/feeds/%.rss.xml,<span class="k">$(</span>notdir <span class="k">$(</span>tag-caches<span class="k">)))</span>
<span class="nv">non-post-sources</span> <span class="o">:=</span> <span class="k">$(</span>wildcard <span class="k">$(</span>src<span class="k">)</span>/non-posts/*.md<span class="k">)</span>
<span class="nv">non-post-htmls</span> <span class="o">:=</span> <span class="k">$(</span>patsubst %.md,<span class="k">$(</span>www<span class="k">)</span>/%.html,<span class="k">$(</span>notdir <span class="k">$(</span>non-post-sources<span class="k">)))</span>
</pre></div>
</td></tr></tbody></table>
</div>
<p>Then, for example, a rule like this is how we turn a <code>.md</code> source into an intermediate <code>.rktd</code> cached file:</p>
<div class="brush: make">
<table class="sourcetable">
<tbody>
<tr>
<td class="linenos">
<div class="linenodiv">
<pre>1
2</pre></div></td>
<td class="code">
<div class="source">
<pre><span></span><span class="nf">$(cache)/%.rktd</span><span class="o">:</span> <span class="k">$(</span><span class="nv">src</span><span class="k">)</span>/<span class="n">posts</span>/%.<span class="n">md</span>
<span class="k">$(</span>make-post-cache<span class="k">)</span> $< <span class="k">$(</span>abspath <span class="k">$(</span>cache<span class="k">)</span>/tags<span class="k">)</span> <span class="nv">$@</span>
</pre></div>
</td></tr></tbody></table>
</div>
<p>and that into an <code>.html</code> file:</p>
<div class="brush: make">
<table class="sourcetable">
<tbody>
<tr>
<td class="linenos">
<div class="linenodiv">
<pre>1
2</pre></div></td>
<td class="code">
<div class="source">
<pre><span></span><span class="nf">$(www)/%.html</span><span class="o">:</span> <span class="k">$(</span><span class="nv">cache</span><span class="k">)</span>/%.<span class="n">rktd</span> <span class="n">rkt</span>/<span class="n">page</span>-<span class="n">xexpr</span>.<span class="n">rkt</span>
<span class="k">$(</span>make-post-html<span class="k">)</span> $< <span class="k">$(</span>www<span class="k">)</span> <span class="nv">$@</span>
</pre></div>
</td></tr></tbody></table>
</div>
<p>What is this business about an “intermediate post cached file”? Well, the overall model is a two-stage “compile and link” sort of metaphor, similar to Frog. Post <code>.md</code> sources only define the “body” of a post page, e.g. the <code><article></code> element that will go somewhere in a page. Building that can be semi-expensive when you do enhancements like syntax-highlighting and automatic links to Racket documentation. It’s nice not to need to redo that, simply because some other element on the containing page changes. Also, each post has one or more tags, so the first pass is a chance to build an index of tags to posts, which the second pass can use to make feed files and an index page for each tag.</p>
<p>The little commands that the <code>Makefile</code> calls are defined in in a <code>rkt/</code> subdirectory. Some of them <code>require</code> modules from Frog. Others are copypasta with some adaptation. Any of these files that could be truly generic — i.e. won’t elicit a pull-request someday to get different blog behavior — I’d like to move eventually into a “tadpole” package. The remaining code should go in some repo — <em>not</em> a package — as example code. That is, you could copy it — probably not even fork it — and just hack away. It is your blog, so it is your program.</p>
<p>So far I’m enjoying this new approach.</p>
<p>Again, I still plan to maintain Frog, in terms of fixing bugs and keeping it working on newer versions of Racket. Already in the past couple years I’ve been super reluctant to add new features, and haven’t. All I’m saying here is that I plan to continue that not-doing.</p>
<footer></footer></article></div>Thread namesurn:www-greghendershott-com:-2018-11-thread-names.html2018-11-02T00:10:21Z2018-11-02T00:10:21ZGreg Hendershott
<div>
<article>
<header>
<h1>Thread names</h1>
<p class="date-and-tags">
<time datetime="2018-11-01" pubdate="true">2018-11-01</time> :: <span class="tags"><a href="/tags/Racket.html">Racket</a></span></p></header>
<p>Sometimes people want Racket <code><a href="http://docs.racket-lang.org/reference/threads.html#(def._((quote._~23~25kernel)._thread))" style="color: inherit">thread</a></code>s to have useful names — for example to show in logger output. Here is one way to do it.</p><!-- more-->
<p>When you call <code>thread</code> it returns a <em>thread descriptor</em>.</p>
<p>You may also get the thread descriptor of the current thread using <code><a href="http://docs.racket-lang.org/reference/threads.html#(def._((quote._~23~25kernel)._current-thread))" style="color: inherit">current-thread</a></code>.</p>
<p>The <code><a href="http://docs.racket-lang.org/reference/Writing.html#(def._((quote._~23~25kernel)._print))" style="color: inherit">print</a></code>ed representation of a thread descriptor includes the <code><a href="http://docs.racket-lang.org/reference/inspectors.html#(def._((quote._~23~25kernel)._object-name))" style="color: inherit">object-name</a></code> of its thunk procedure.</p>
<p>Often the thunk is an anonymous function. In that case the function object-name is something like <code>/path/to/file.rkt:1:3</code> — so the thread descriptor prints as <code>#<thread:/path/to/file.rkt:1:3></code>.</p>
<div class="brush: racket">
<table class="sourcetable">
<tbody>
<tr>
<td class="linenos">
<div class="linenodiv">
<pre>1</pre></div></td>
<td class="code">
<div class="source">
<pre><span></span><span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/threads.html#(def._((quote._~23~25kernel)._thread))" style="color: inherit">thread</a></span> <span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/lambda.html#(form._((lib._racket/private/base..rkt)._~ce~bb))" style="color: inherit">λ</a></span> <span class="p">()</span> <span class="no">#f</span><span class="p">))</span> <span class="c1">;=> #<thread:/tmp/thread.rkt:9:8></span>
</pre></div>
</td></tr></tbody></table>
</div>
<p>It is more interesting when you name the thunk:</p>
<div class="brush: racket">
<table class="sourcetable">
<tbody>
<tr>
<td class="linenos">
<div class="linenodiv">
<pre>1
2</pre></div></td>
<td class="code">
<div class="source">
<pre><span></span><span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/define.html#(form._((lib._racket/private/base..rkt)._define))" style="color: inherit">define</a></span> <span class="p">(</span><span class="n">foo</span><span class="p">)</span> <span class="no">#f</span><span class="p">)</span>
<span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/threads.html#(def._((quote._~23~25kernel)._thread))" style="color: inherit">thread</a></span> <span class="n">foo</span><span class="p">)</span> <span class="c1">;=> #<thread:foo></span>
</pre></div>
</td></tr></tbody></table>
</div>
<p>You can also use <code>object-name</code> on the thread descriptor to extract the thunk’s <code>object-name</code> as a symbol:</p>
<div class="brush: racket">
<table class="sourcetable">
<tbody>
<tr>
<td class="linenos">
<div class="linenodiv">
<pre>1
2</pre></div></td>
<td class="code">
<div class="source">
<pre><span></span><span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/inspectors.html#(def._((quote._~23~25kernel)._object-name))" style="color: inherit">object-name</a></span> <span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/threads.html#(def._((quote._~23~25kernel)._thread))" style="color: inherit">thread</a></span> <span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/lambda.html#(form._((lib._racket/private/base..rkt)._~ce~bb))" style="color: inherit">λ</a></span> <span class="p">()</span> <span class="no">#f</span><span class="p">)))</span> <span class="c1">;=> '/tmp/thread.rkt:18:21</span>
<span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/inspectors.html#(def._((quote._~23~25kernel)._object-name))" style="color: inherit">object-name</a></span> <span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/threads.html#(def._((quote._~23~25kernel)._thread))" style="color: inherit">thread</a></span> <span class="n">foo</span><span class="p">))</span> <span class="c1">;=> 'foo</span>
</pre></div>
</td></tr></tbody></table>
</div>
<p>What if you start 10 threads that all use the same thunk procedure? You can use <code><a href="http://docs.racket-lang.org/reference/procedures.html#(def._((lib._racket/private/base..rkt)._procedure-rename))" style="color: inherit">procedure-rename</a></code> to rename the thunk each time:</p>
<div class="brush: racket">
<table class="sourcetable">
<tbody>
<tr>
<td class="linenos">
<div class="linenodiv">
<pre>1
2</pre></div></td>
<td class="code">
<div class="source">
<pre><span></span><span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/threads.html#(def._((quote._~23~25kernel)._thread))" style="color: inherit">thread</a></span> <span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/procedures.html#(def._((lib._racket/private/base..rkt)._procedure-rename))" style="color: inherit">procedure-rename</a></span> <span class="n">foo</span> <span class="o">'</span><span class="ss">bar</span><span class="p">))</span> <span class="c1">;=> #<thread:bar></span>
<span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/threads.html#(def._((quote._~23~25kernel)._thread))" style="color: inherit">thread</a></span> <span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/procedures.html#(def._((lib._racket/private/base..rkt)._procedure-rename))" style="color: inherit">procedure-rename</a></span> <span class="n">foo</span> <span class="o">'</span><span class="ss">baz</span><span class="p">))</span> <span class="c1">;=> #<thread:baz></span>
</pre></div>
</td></tr></tbody></table>
</div>
<p>The new name can be any symbol — including one generated at runtime:</p>
<div class="brush: racket">
<table class="sourcetable">
<tbody>
<tr>
<td class="linenos">
<div class="linenodiv">
<pre>1
2
3
4</pre></div></td>
<td class="code">
<div class="source">
<pre><span></span><span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/define.html#(form._((lib._racket/private/base..rkt)._define))" style="color: inherit">define</a></span> <span class="p">(</span><span class="n">now-sym</span><span class="p">)</span>
<span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/symbols.html#(def._((quote._~23~25kernel)._string-~3esymbol))" style="color: inherit">string->symbol</a></span> <span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/strings.html#(def._((lib._racket/format..rkt)._~7ea))" style="color: inherit">~a</a></span> <span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/time.html#(def._((quote._~23~25kernel)._current-inexact-milliseconds))" style="color: inherit">current-inexact-milliseconds</a></span><span class="p">))))</span>
<span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/threads.html#(def._((quote._~23~25kernel)._thread))" style="color: inherit">thread</a></span> <span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/procedures.html#(def._((lib._racket/private/base..rkt)._procedure-rename))" style="color: inherit">procedure-rename</a></span> <span class="n">foo</span> <span class="p">(</span><span class="n">now-sym</span><span class="p">)))</span> <span class="c1">;=> ex: #<thread:1541116698791.029></span>
<span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/threads.html#(def._((quote._~23~25kernel)._thread))" style="color: inherit">thread</a></span> <span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/procedures.html#(def._((lib._racket/private/base..rkt)._procedure-rename))" style="color: inherit">procedure-rename</a></span> <span class="n">foo</span> <span class="p">(</span><span class="n">now-sym</span><span class="p">)))</span> <span class="c1">;=> ex: #<thread:1541116698791.18></span>
</pre></div>
</td></tr></tbody></table>
</div>
<p>Of course, you can include in the generated name any interesting identifying information from your problem domain — such as an “ID” or other details about a “job”, “request”, or “message”.</p>
<p>So something like this:</p>
<div class="brush: racket">
<table class="sourcetable">
<tbody>
<tr>
<td class="linenos">
<div class="linenodiv">
<pre> 1
2
3
4
5
6
7
8
9
10
11
12
13
14</pre></div></td>
<td class="code">
<div class="source">
<pre><span></span><span class="kn">#lang </span><span class="nn">at-exp</span> <span class="n">racket/base</span>
<span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/require.html#(form._((lib._racket/private/base..rkt)._require))" style="color: inherit">require</a></span> <span class="n">racket/format</span><span class="p">)</span>
<span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/logging.html#(form._((lib._racket/private/more-scheme..rkt)._define-logger))" style="color: inherit">define-logger</a></span> <span class="n">thor</span><span class="p">)</span>
<span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/define.html#(form._((lib._racket/private/base..rkt)._define))" style="color: inherit">define</a></span> <span class="p">(</span><span class="n">thor</span><span class="p">)</span>
<span class="p">(</span><span class="n">log-thor-debug</span> <span class="n">@~a</span><span class="p">{</span><span class="n">hammering</span> <span class="n">job</span> <span class="n">@</span><span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/inspectors.html#(def._((quote._~23~25kernel)._object-name))" style="color: inherit">object-name</a></span> <span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/threads.html#(def._((quote._~23~25kernel)._current-thread))" style="color: inherit">current-thread</a></span><span class="p">))}))</span>
<span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/define.html#(form._((lib._racket/private/base..rkt)._define))" style="color: inherit">define</a></span> <span class="p">(</span><span class="n">job-id</span><span class="p">)</span>
<span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/symbols.html#(def._((quote._~23~25kernel)._string-~3esymbol))" style="color: inherit">string->symbol</a></span> <span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/strings.html#(def._((lib._racket/format..rkt)._~7ea))" style="color: inherit">~a</a></span> <span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/time.html#(def._((quote._~23~25kernel)._current-inexact-milliseconds))" style="color: inherit">current-inexact-milliseconds</a></span><span class="p">))))</span>
<span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/threads.html#(def._((quote._~23~25kernel)._thread))" style="color: inherit">thread</a></span> <span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/procedures.html#(def._((lib._racket/private/base..rkt)._procedure-rename))" style="color: inherit">procedure-rename</a></span> <span class="n">thor</span> <span class="p">(</span><span class="n">job-id</span><span class="p">)))</span>
<span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/threads.html#(def._((quote._~23~25kernel)._thread))" style="color: inherit">thread</a></span> <span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/procedures.html#(def._((lib._racket/private/base..rkt)._procedure-rename))" style="color: inherit">procedure-rename</a></span> <span class="n">thor</span> <span class="p">(</span><span class="n">job-id</span><span class="p">)))</span>
</pre></div>
</td></tr></tbody></table>
</div>
<p>Can produce logger output like this:</p>
<div class="brush: sh">
<table class="sourcetable">
<tbody>
<tr>
<td class="linenos">
<div class="linenodiv">
<pre>1
2</pre></div></td>
<td class="code">
<div class="source">
<pre><span></span><span class="o">[</span> debug<span class="o">]</span> thor: hammering job <span class="m">1541119025888</span>.965
<span class="o">[</span> debug<span class="o">]</span> thor: hammering job <span class="m">1541119025889</span>.021
</pre></div>
</td></tr></tbody></table>
</div>
<p>In conclusion, it is possible for threads to have unique names in Racket — provided you’re OK giving unique names to the thunks run by the theads.</p>
<footer></footer></article></div>racket-modeurn:www-greghendershott-com:-2018-10-racket-mode.html2018-10-03T04:00:00Z2018-10-03T04:00:00ZGreg Hendershott
<div>
<article>
<header>
<h1>racket-mode</h1>
<p class="date-and-tags">
<time datetime="2018-10-03" pubdate="true">2018-10-03</time> :: <span class="tags"><a href="/tags/Racket.html">Racket</a>, <a href="/tags/Emacs.html">Emacs</a></span></p></header>
<p>Even though it’s been one of my most time-consuming projects, I’ve hardly blogged about <a href="https://github.com/greghendershott/racket-mode">racket-mode</a> — an <a href="https://www.gnu.org/software/emacs/">Emacs</a> major mode for <a href="https://www.racket-lang.org">Racket</a>. To change that, here’s a post giving an overview of how racket-mode works, as well as a look at how it might grow someday.</p><!-- more-->
<h1 id="cast-of-characters">Cast of characters</h1>
<p>racket-mode consists of two main parts:</p>
<ul>
<li>
<p>An <a href="https://www.gnu.org/software/emacs/manual/html_node/eintr/index.html">Emacs Lisp</a> “front end”. This implements various major and minor modes, the main two of which are:</p>
<ul>
<li>
<p><code>racket-mode</code> for editing <code>.rkt</code> files</p></li>
<li>
<p><code>racket-repl-mode</code> for a REPL</p></li></ul></li>
<li>A Racket “back end”. This is a <code>racket</code> process that runs the back end code, which in turn runs your program and implements “commands”, as discussed below.</li></ul>
<p>This mix of Emacs Lisp and Racket code is delivered as an Emacs package (e.g. via <a href="https://melpa.org">MELPA</a>), where the Racket files come along for the ride.<sup><a href="#racket-mode-footnote-1-definition" name="racket-mode-footnote-1-return">1</a></sup></p>
<h1 id="crossing-the-chasm">Crossing the chasm</h1>
<p>The Emacs front end (“client”) sends command requests to the Racket back end (“server”) which sends command responses. Exactly <em>how</em> this happens is the aspect of racket-mode that has changed the most over the years.</p>
<h2 id="historical">Historical</h2>
<p>The very earliest version of racket-mode was a very thin wrapper around <a href="https://docs.racket-lang.org/xrepl/">XREPL</a>, which lets you enter commands prefixed by a comma<sup><a href="#racket-mode-footnote-2-definition" name="racket-mode-footnote-2-return">2</a></sup> like <code>,enter <filename></code>. So racket-mode would mostly just <code>comint-send-string</code> a few of these commands (invisibly type them into the buffer on your behalf).</p>
<p>But.</p>
<ul>
<li>
<p>What if the user’s program does <code>read</code> or <code>read-line</code>? If we send commands to the back end, the user’s program will get them.</p></li>
<li>
<p>Later, command <em>responses</em> became more important. For example, racket-mode’s <code>racket-describe</code> command returns HTML (from Racket’s documentation) to be shown in an Emacs buffer. Well, occasionally the HTML might contain something like <code>"\ndiv>"</code> — breaking code trying to read a response up to the next REPL prompt.</p></li></ul>
<p>Long story short: If you like edge cases, you’ll love multiplexing textual command I/O with user program I/O. That approach is an easy way to start casually. But it’s actually easier long-term to move the command I/O “out-of-band”, when possible. It sucks to demux.</p>
<h2 id="today">Today</h2>
<p>Early Summer 2018 I again changed how command requests and responses work.</p>
<p>The Racket back end starts a TCP “command server”, and the Emacs front end connects to it.</p>
<p>Although the connection method is TCP, the protocol isn’t HTTP. Instead, the command server:</p>
<ul>
<li>
<p>Accepts a single TCP connection at a time (multiple requests and responses are sent over the one connection)</p></li>
<li>
<p>Uses a subset of valid Emacs Lisp s-expressions for requests and responses.</p></li></ul>
<p>Command requests are <code>(nonce command param ...)</code>.</p>
<p>A thread is spun off to handle each request, so that a long-running command won’t block others. The <code>nonce</code> supplied with the request is returned with the response, so that the client can match the response with the request.<sup><a href="#racket-mode-footnote-3-definition" name="racket-mode-footnote-3-return">3</a></sup></p>
<p>Command responses are either <code>(nonce 'ok sexp ...+)</code> or <code>(nonce 'error
"message")</code>.</p>
<p>Here’s a code snippet to give a rough sense of the commands:</p>
<div class="brush: racket">
<table class="sourcetable">
<tbody>
<tr>
<td class="linenos">
<div class="linenodiv">
<pre> 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24</pre></div></td>
<td class="code">
<div class="source">
<pre><span></span> <span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/match.html#(form._((lib._racket/match..rkt)._match))" style="color: inherit">match</a></span> <span class="n">sexpr</span>
<span class="p">[</span><span class="o">`</span><span class="p">(</span><span class="ss">run</span> <span class="o">,</span><span class="n">what</span> <span class="o">,</span><span class="n">mem</span> <span class="o">,</span><span class="n">pp?</span> <span class="o">,</span><span class="n">ctx</span> <span class="o">,</span><span class="n">args</span> <span class="o">,</span><span class="n">dbg</span><span class="p">)</span> <span class="p">(</span><span class="n">run</span> <span class="n">what</span> <span class="n">mem</span> <span class="n">pp?</span> <span class="n">ctx</span> <span class="n">args</span> <span class="n">dbg</span><span class="p">)]</span>
<span class="p">[</span><span class="o">`</span><span class="p">(</span><span class="ss">path+md5</span><span class="p">)</span> <span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/pairs.html#(def._((quote._~23~25kernel)._cons))" style="color: inherit">cons</a></span> <span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/if.html#(form._((lib._racket/private/letstx-scheme..rkt)._or))" style="color: inherit">or</a></span> <span class="n">path</span> <span class="o">'</span><span class="ss">top</span><span class="p">)</span> <span class="n">md5</span><span class="p">)]</span>
<span class="p">[</span><span class="o">`</span><span class="p">(</span><span class="ss">syms</span><span class="p">)</span> <span class="p">(</span><span class="n">syms</span><span class="p">)]</span>
<span class="p">[</span><span class="o">`</span><span class="p">(</span><span class="ss">def</span> <span class="o">,</span><span class="n">str</span><span class="p">)</span> <span class="p">(</span><span class="n">find-definition</span> <span class="n">str</span><span class="p">)]</span>
<span class="p">[</span><span class="o">`</span><span class="p">(</span><span class="ss">mod</span> <span class="o">,</span><span class="n">sym</span><span class="p">)</span> <span class="p">(</span><span class="n">find-module</span> <span class="n">sym</span> <span class="n">maybe-mod</span><span class="p">)]</span>
<span class="p">[</span><span class="o">`</span><span class="p">(</span><span class="ss">describe</span> <span class="o">,</span><span class="n">str</span><span class="p">)</span> <span class="p">(</span><span class="n">describe</span> <span class="n">str</span><span class="p">)]</span>
<span class="p">[</span><span class="o">`</span><span class="p">(</span><span class="ss">doc</span> <span class="o">,</span><span class="n">str</span><span class="p">)</span> <span class="p">(</span><span class="n">doc</span> <span class="n">str</span><span class="p">)]</span>
<span class="p">[</span><span class="o">`</span><span class="p">(</span><span class="ss">type</span> <span class="o">,</span><span class="n">v</span><span class="p">)</span> <span class="p">(</span><span class="n">type</span> <span class="n">v</span><span class="p">)]</span>
<span class="p">[</span><span class="o">`</span><span class="p">(</span><span class="ss">macro-stepper</span> <span class="o">,</span><span class="n">str</span> <span class="o">,</span><span class="n">into-base?</span><span class="p">)</span> <span class="p">(</span><span class="n">macro-stepper</span> <span class="n">str</span> <span class="n">into-base?</span><span class="p">)]</span>
<span class="p">[</span><span class="o">`</span><span class="p">(</span><span class="ss">macro-stepper/next</span><span class="p">)</span> <span class="p">(</span><span class="n">macro-stepper/next</span><span class="p">)]</span>
<span class="p">[</span><span class="o">`</span><span class="p">(</span><span class="ss">requires/tidy</span> <span class="o">,</span><span class="n">reqs</span><span class="p">)</span> <span class="p">(</span><span class="n">requires/tidy</span> <span class="n">reqs</span><span class="p">)]</span>
<span class="p">[</span><span class="o">`</span><span class="p">(</span><span class="ss">requires/trim</span> <span class="o">,</span><span class="n">path-str</span> <span class="o">,</span><span class="n">reqs</span><span class="p">)</span> <span class="p">(</span><span class="n">requires/trim</span> <span class="n">path-str</span> <span class="n">reqs</span><span class="p">)]</span>
<span class="p">[</span><span class="o">`</span><span class="p">(</span><span class="ss">requires/base</span> <span class="o">,</span><span class="n">path-str</span> <span class="o">,</span><span class="n">reqs</span><span class="p">)</span> <span class="p">(</span><span class="n">requires/base</span> <span class="n">path-str</span> <span class="n">reqs</span><span class="p">)]</span>
<span class="p">[</span><span class="o">`</span><span class="p">(</span><span class="ss">find-collection</span> <span class="o">,</span><span class="n">str</span><span class="p">)</span> <span class="p">(</span><span class="n">find-collection</span> <span class="n">str</span><span class="p">)]</span>
<span class="p">[</span><span class="o">`</span><span class="p">(</span><span class="ss">get-profile</span><span class="p">)</span> <span class="p">(</span><span class="n">get-profile</span><span class="p">)]</span>
<span class="p">[</span><span class="o">`</span><span class="p">(</span><span class="ss">get-uncovered</span><span class="p">)</span> <span class="p">(</span><span class="n">get-uncovered</span> <span class="n">path</span><span class="p">)]</span>
<span class="p">[</span><span class="o">`</span><span class="p">(</span><span class="ss">check-syntax</span> <span class="o">,</span><span class="n">path-str</span><span class="p">)</span> <span class="p">(</span><span class="n">check-syntax</span> <span class="n">path-str</span><span class="p">)]</span>
<span class="p">[</span><span class="o">`</span><span class="p">(</span><span class="ss"><a href="http://docs.racket-lang.org/reference/eval.html#(def._((quote._~23~25kernel)._eval))" style="color: inherit">eval</a></span> <span class="o">,</span><span class="n">v</span><span class="p">)</span> <span class="p">(</span><span class="n">eval-command</span> <span class="n">v</span><span class="p">)]</span>
<span class="p">[</span><span class="o">`</span><span class="p">(</span><span class="ss">repl-submit?</span> <span class="o">,</span><span class="n">str</span> <span class="o">,</span><span class="n">eos?</span><span class="p">)</span> <span class="p">(</span><span class="n">repl-submit?</span> <span class="n">submit-pred</span> <span class="n">str</span> <span class="n">eos?</span><span class="p">)]</span>
<span class="p">[</span><span class="o">`</span><span class="p">(</span><span class="ss">debug-eval</span> <span class="o">,</span><span class="n">src</span> <span class="o">,</span><span class="n">l</span> <span class="o">,</span><span class="n">c</span> <span class="o">,</span><span class="n">p</span> <span class="o">,</span><span class="n">code</span><span class="p">)</span> <span class="p">(</span><span class="n">debug-eval</span> <span class="n">src</span> <span class="n">l</span> <span class="n">c</span> <span class="n">p</span> <span class="n">code</span><span class="p">)]</span>
<span class="p">[</span><span class="o">`</span><span class="p">(</span><span class="ss">debug-resume</span> <span class="o">,</span><span class="n">v</span><span class="p">)</span> <span class="p">(</span><span class="n">debug-resume</span> <span class="n">v</span><span class="p">)]</span>
<span class="p">[</span><span class="o">`</span><span class="p">(</span><span class="ss">debug-disable</span><span class="p">)</span> <span class="p">(</span><span class="n">debug-disable</span><span class="p">)]</span>
<span class="p">[</span><span class="o">`</span><span class="p">(</span><span class="ss"><a href="http://docs.racket-lang.org/reference/Exiting.html#(def._((quote._~23~25kernel)._exit))" style="color: inherit">exit</a></span><span class="p">)</span> <span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/Exiting.html#(def._((quote._~23~25kernel)._exit))" style="color: inherit">exit</a></span><span class="p">)])</span>
</pre></div>
</td></tr></tbody></table>
</div>
<p>This approach also makes it reasonably simple for the Emacs Lisp front end to issue some commands “asynchronously”. In fact the lowest-level command function is async:</p>
<div class="brush: elisp">
<table class="sourcetable">
<tbody>
<tr>
<td class="linenos">
<div class="linenodiv">
<pre> 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18</pre></div></td>
<td class="code">
<div class="source">
<pre><span></span><span class="p">(</span><span class="k">defvar</span> <span class="nv">racket--cmd-nonce->callback</span> <span class="p">(</span><span class="nf">make-hash-table</span> <span class="nb">:test</span> <span class="ss">'eq</span><span class="p">)</span>
<span class="s">"A hash from nonce to callback function."</span><span class="p">)</span>
<span class="p">(</span><span class="k">defvar</span> <span class="nv">racket--cmd-nonce</span> <span class="mi">0</span>
<span class="s">"Increments for each command request we send."</span><span class="p">)</span>
<span class="p">(</span><span class="nb">defun</span> <span class="nv">racket--cmd/async-raw</span> <span class="p">(</span><span class="nv">command-sexpr</span> <span class="kp">&optional</span> <span class="nv">callback</span><span class="p">)</span>
<span class="s">"Send COMMAND-SEXPR and return. Later call CALLBACK with the response sexp.</span>
<span class="s">If CALLBACK is not supplied or nil, defaults to </span><span class="ss">`ignore'</span><span class="s">."</span>
<span class="p">(</span><span class="nv">racket--repl-ensure-buffer-and-process</span> <span class="no">nil</span><span class="p">)</span>
<span class="p">(</span><span class="nv">racket--cmd-connect-finish</span><span class="p">)</span>
<span class="p">(</span><span class="nb">cl-incf</span> <span class="nv">racket--cmd-nonce</span><span class="p">)</span>
<span class="p">(</span><span class="nb">when</span> <span class="p">(</span><span class="k">and</span> <span class="nv">callback</span>
<span class="p">(</span><span class="nv">not</span> <span class="p">(</span><span class="nf">equal</span> <span class="nv">callback</span> <span class="nf">#'</span><span class="nv">ignore</span><span class="p">)))</span>
<span class="p">(</span><span class="nf">puthash</span> <span class="nv">racket--cmd-nonce</span> <span class="nv">callback</span> <span class="nv">racket--cmd-nonce->callback</span><span class="p">))</span>
<span class="p">(</span><span class="nf">process-send-string</span> <span class="nv">racket--cmd-proc</span>
<span class="p">(</span><span class="nf">format</span> <span class="s">"%S\n"</span> <span class="p">(</span><span class="nf">cons</span> <span class="nv">racket--cmd-nonce</span>
<span class="nv">command-sexpr</span><span class="p">))))</span>
</pre></div>
</td></tr></tbody></table>
</div>
<p>Code that doesn’t need to examine error responses can use a simplified wrapper:</p>
<div class="brush: elisp">
<table class="sourcetable">
<tbody>
<tr>
<td class="linenos">
<div class="linenodiv">
<pre> 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15</pre></div></td>
<td class="code">
<div class="source">
<pre><span></span><span class="p">(</span><span class="nb">defun</span> <span class="nv">racket--cmd/async</span> <span class="p">(</span><span class="nv">command-sexpr</span> <span class="kp">&optional</span> <span class="nv">callback</span><span class="p">)</span>
<span class="s">"You probably want to use this instead of </span><span class="ss">`racket--cmd/async-raw'</span><span class="s">.</span>
<span class="s">CALLBACK is only called for 'ok responses, with (ok v ...)</span>
<span class="s">unwrapped to (v ...). 'error responses are handled here."</span>
<span class="p">(</span><span class="k">let</span> <span class="p">((</span><span class="nv">buf</span> <span class="p">(</span><span class="nf">current-buffer</span><span class="p">)))</span>
<span class="p">(</span><span class="nv">racket--cmd/async-raw</span>
<span class="nv">command-sexpr</span>
<span class="p">(</span><span class="k">if</span> <span class="nv">callback</span>
<span class="p">(</span><span class="nb">lambda</span> <span class="p">(</span><span class="nv">response</span><span class="p">)</span>
<span class="p">(</span><span class="nb">pcase</span> <span class="nv">response</span>
<span class="p">(</span><span class="o">`</span><span class="p">(</span><span class="nv">ok</span> <span class="o">,</span><span class="nv">v</span><span class="p">)</span> <span class="p">(</span><span class="nb">with-current-buffer</span> <span class="nv">buf</span> <span class="p">(</span><span class="nf">funcall</span> <span class="nv">callback</span> <span class="nv">v</span><span class="p">)))</span>
<span class="p">(</span><span class="o">`</span><span class="p">(</span><span class="ne">error</span> <span class="o">,</span><span class="nv">m</span><span class="p">)</span> <span class="p">(</span><span class="nf">message</span> <span class="s">"%s"</span> <span class="nv">m</span><span class="p">))</span>
<span class="p">(</span><span class="nv">v</span> <span class="p">(</span><span class="nf">message</span> <span class="s">"Unknown command response: %S"</span> <span class="nv">v</span><span class="p">))))</span>
<span class="nf">#'</span><span class="nv">ignore</span><span class="p">))))</span>
</pre></div>
</td></tr></tbody></table>
</div>
<p>Code that needs to wait synchronously for a response can use another wrapper:</p>
<div class="brush: elisp">
<table class="sourcetable">
<tbody>
<tr>
<td class="linenos">
<div class="linenodiv">
<pre> 1
2
3
4
5
6
7
8
9
10
11
12
13
14</pre></div></td>
<td class="code">
<div class="source">
<pre><span></span><span class="p">(</span><span class="nb">defun</span> <span class="nv">racket--cmd/await</span> <span class="p">(</span><span class="nv">command-sexpr</span><span class="p">)</span>
<span class="s">"Send COMMAND-SEXPR. Await and return an 'ok response value, or raise </span><span class="ss">`error'</span><span class="s">."</span>
<span class="p">(</span><span class="k">let*</span> <span class="p">((</span><span class="nv">awaiting</span> <span class="ss">'RACKET-REPL-AWAITING</span><span class="p">)</span>
<span class="p">(</span><span class="nv">response</span> <span class="nv">awaiting</span><span class="p">))</span>
<span class="p">(</span><span class="nv">racket--cmd/async-raw</span> <span class="nv">command-sexpr</span>
<span class="p">(</span><span class="nb">lambda</span> <span class="p">(</span><span class="nv">v</span><span class="p">)</span> <span class="p">(</span><span class="k">setq</span> <span class="nv">response</span> <span class="nv">v</span><span class="p">)))</span>
<span class="p">(</span><span class="nb">with-timeout</span> <span class="p">(</span><span class="nv">racket-command-timeout</span>
<span class="p">(</span><span class="ne">error</span> <span class="s">"racket-command process timeout"</span><span class="p">))</span>
<span class="p">(</span><span class="k">while</span> <span class="p">(</span><span class="nf">eq</span> <span class="nv">response</span> <span class="nv">awaiting</span><span class="p">)</span>
<span class="p">(</span><span class="nf">accept-process-output</span> <span class="no">nil</span> <span class="mf">0.001</span><span class="p">))</span>
<span class="p">(</span><span class="nb">pcase</span> <span class="nv">response</span>
<span class="p">(</span><span class="o">`</span><span class="p">(</span><span class="nv">ok</span> <span class="o">,</span><span class="nv">v</span><span class="p">)</span> <span class="nv">v</span><span class="p">)</span>
<span class="p">(</span><span class="o">`</span><span class="p">(</span><span class="ne">error</span> <span class="o">,</span><span class="nv">m</span><span class="p">)</span> <span class="p">(</span><span class="ne">error</span> <span class="s">"%s"</span> <span class="nv">m</span><span class="p">))</span>
<span class="p">(</span><span class="nv">v</span> <span class="p">(</span><span class="ne">error</span> <span class="s">"Unknown command response: %S"</span> <span class="nv">v</span><span class="p">))))))</span>
</pre></div>
</td></tr></tbody></table>
</div>
<p>In other words, asynchronous is the baseline. Synchronous is the special case where the callback <code>setq</code>s the response and the main Emacs thread polls for it.</p>
<h1 id="what-does-it-mean-to-run-a-file">What does it mean to “run” a file</h1>
<p>The main command is <code>run</code>, which causes it to run a specific <code>.rkt</code> file. “Run” means something similar to what it does in <a href="https://docs.racket-lang.org/drracket/index.html">DrRacket</a>: <code>dynamic-require</code> a relative module path and use <code>module->namespace</code> so that you are “inside” the module and can see all its module-level definitions (even those it does not <code>provide</code>). Then it does a <code>read-eval-print-loop</code>, which is the REPL you see in the <code>racket-repl-mode</code> buffer.</p>
<p>Importantly, each run uses a fresh custodian and namespace. Therefore each run discards any changes you made only in the REPL and “resets” you to a namespace corresponding to the <code>.rkt</code> file. In other words, the single source of truth is your source file.</p>
<p>One handy nuance: If, when you run, point<sup><a href="#racket-mode-footnote-4-definition" name="racket-mode-footnote-4-return">4</a></sup> is inside a submodule at any depth, you enter that submodule. So for example you can put point within a <code>(module+ test ___)</code> form and
<kbd>C-c C-c</kbd>. Not only will this run your tests, the REPL will be “inside” the <code>test</code> submodule so you can explore and experiment more. Or you can run a <code>main</code>, or any other, submodule.</p>
<p>Of course there are many wrinkles:</p>
<ul>
<li>
<p>In some cases we “instrument” the module to support features like profiling, test coverage, or step debugging.</p></li>
<li>
<p>We look for and run <code>configure-runtime</code>s or <code>get-info</code>s supplied by a <code>#lang</code>.</p></li>
<li>
<p>We warn when a <code>#lang</code> doesn’t supply <code>#%top-interaction</code>.</p></li>
<li>
<p>We load <code>racket/gui</code> lazily. (That way, you don’t get a pointless GUI frame window while working on textual programs.)</p></li>
<li>
<p>We maybe set up custom print handlers to use pretty-print.</p></li>
<li>
<p>If the file contains syntax errors, we try to distill it to a “skeleton” of things that create runtime, module-level bindings, i.e. <code>require</code>s and <code>define</code>s. You can think of this as replacing all <code>(define id rhs)</code> with <code>(define id (void))</code>. Why? That way, we get a namespace with the definition identifiers available to things like auto-complete.</p></li>
<li>
<p>And on and on.</p></li></ul>
<h1 id="summer-two-step">Summer two step</h1>
<p>Over the summer of 2018, I added a couple features under the theme of “stepping”.</p>
<h2 id="macro-step-expander">Macro step expander</h2>
<p>A new <code>racket-stepper-mode</code> presents a consistent <code>diff -U 3</code> format UI for both:</p>
<ul>
<li>
<p><code>expand-once</code>-ing expressions</p></li>
<li>
<p>whole-file expansion using <a href="https://docs.racket-lang.org/macro-debugger/index.html#(mod-path._macro-debugger%2Fstepper-text)"><code>macro-debugger/stepper-text</code></a> (although the UI is not as ambitious as the fantastic macro stepper in DrRacket)</p></li></ul>
<p>Whole-file expansion potentially can take many seconds. So this is an example where it’s good to have the ability for command responses to come as an asynchronous callback. Emacs isn’t “frozen” while waiting.</p>
<h2 id="step-debugger">Step debugger</h2>
<p>Also new is an interactive step debugger. I won’t repeat the <a href="https://github.com/greghendershott/racket-mode/commit/2b1c7d476dc71b1707fd5222f963ab6509e50805">commit message</a> or the <a href="https://github.com/greghendershott/racket-mode/blob/master/Reference.md#racket-debug-mode">documentation</a> here. I’ll just mention that, as the commit message says, I had started to work on this several years ago but decided not to merge it. The new async command protocol was one factor that made it easier. Also, by labeling it “experimental”, I was less worried should it turn out to be too much hassle to support long-term.</p>
<p>I’m curious to see how much people actually use it. I’ve found that I want an interactive debugger <em>much</em> less often writing mostly functional Racket than I did when writing imperative C/C++. With Racket, a step debugger feels more like a nice-to-have than a must-have. Even so, while working in Emacs Lisp for racket-mode, sometimes I have found <a href="https://www.gnu.org/software/emacs/manual/html_node/eintr/edebug.html"><code>edebug</code></a> to be handy. So we’ll see.</p>
<p><img src="/img/racket-mode-step-debugger.gif" alt="racket-mode step debugger" class="screenshot" /></p>
<h1 id="future">Future</h1>
<p>Modulo bugs and some UX refinements, I’d say that racket-mode has reached the point where it’s a pretty good "#lang-racket-mode". That is, it’s pretty good for working with s-expression <code>#lang</code>s like <code>racket</code>, <code>racket/base</code>, <code>typed/racket</code>, and so on.</p>
<p>However it does not attempt to be a “racket-is-a-programming-language-programming-language-mode” like DrRacket. Maybe that is where it should head next. If so, I’m aware of three main areas, only one of which is done.</p>
<h2 id="submitted-for-your-approval-in-the-repl-zone">Submitted for your approval, in the REPL zone</h2>
<p>A <code>#lang</code> may supply a <a href="https://docs.racket-lang.org/tools/lang-languages-customization.html#%28elem._drracket~3asubmit-predicate%29"><code>drracket:submit-predicate</code></a> to say whether some chunk of text is a complete expression. In other words, when a user hits
<kbd>ENTER</kbd> in the REPL, should this be sent for evaluation, or, simply insert a newline and wait for them to supply more?</p>
<p>As of a couple months ago, racket-mode does actually look for and use this.</p>
<h2 id="syntax-highlighting">Syntax highlighting</h2>
<p>“Font-lock” is what Emacs calls the process of changing the appearance of text — such as syntax highlighting for programming modes. Mostly this is done via regular expressions, although an arbitrary function can be supplied to do both fancier matching and altering.</p>
<p>Traditionally font-lock distinguishes between language elements that are “keywords” vs. “builtins”. When it comes to Racket, this is sort of a weird distinction that I handled <a href="https://github.com/greghendershott/racket-mode/blob/4440f34bc892ce6ad1036b2123e5f6010872423d/racket/keywords.rkt#L5-L36">ad hoc</a>.</p>
<p>DrRacket takes a different approach. Each <code>#lang</code> may supply a <a href="https://docs.racket-lang.org/tools/lang-languages-customization.html#%28part._.Syntax_.Coloring%29"><code>color-lexer</code></a>. Clearly this is the Correct Way.</p>
<p>And yet. One great way to make Emacs feel sluggish is to do font-lock poorly. To-date, I’ve been worried about the performance of trying to use <code>color-lexer</code>s, which are <a href="https://docs.racket-lang.org/framework/Color.html"><code>color:text<%></code></a> interfaces. How will this perform marshaled over the TCP connection? Will this force racket-mode to use <code>racket/gui</code> always, instead of only when user programs need it?</p>
<h2 id="indentation">Indentation</h2>
<p>Similarly, racket-mode could someday use a <code>#lang</code>-supplied <a href="https://docs.racket-lang.org/tools/lang-languages-customization.html#(part._.Indentation)"><code>drracket:indentation</code></a> — and similarly, naive indentation can bring Emacs to its knees.</p>
<p>Currently racket-mode indents using Emacs Lisp that is mostly hardcoded with special handling for known macros<sup><a href="#racket-mode-footnote-5-definition" name="racket-mode-footnote-5-return">5</a></sup> in some popular <code>#lang</code>s.</p>
<p>I say “mostly hardcoded” because it <em>is</em> extensible in the tradition of lisp-mode and scheme-mode: racket-mode looks for a <a href="https://github.com/greghendershott/racket-mode/blob/master/Reference.md#racket-indent-line"><code>'racket-indent-function</code></a> property on an Emacs Lisp symbol of the same name as the Racket macro. racket-mode sets up some of these itself. You can add your own in your Emacs init file:</p>
<div class="brush: elisp">
<table class="sourcetable">
<tbody>
<tr>
<td class="linenos">
<div class="linenodiv">
<pre>1</pre></div></td>
<td class="code">
<div class="source">
<pre><span></span><span class="p">(</span><span class="nf">put</span> <span class="ss">'racket-macro-name</span> <span class="ss">'racket-indent-function</span> <span class="nv">indent-value</span><span class="p">)</span>
</pre></div>
</td></tr></tbody></table>
</div>
<p>You can also do this using Emacs <a href="https://www.gnu.org/software/emacs/manual/html_node/emacs/Directory-Variables.html">Directory Variables</a> or in <code>.rkt</code> files using Emacs <a href="https://www.gnu.org/software/emacs/manual/html_node/emacs/Specifying-File-Variables.html#Specifying-File-Variables">File Variables</a>. An example of the latter setting up indentation for some <a href="https://docs.racket-lang.org/sql/index.html"><code>sql</code></a> package syntax:</p>
<div class="brush: racket">
<table class="sourcetable">
<tbody>
<tr>
<td class="linenos">
<div class="linenodiv">
<pre>1
2
3
4
5
6</pre></div></td>
<td class="code">
<div class="source">
<pre><span></span><span class="c1">;; Local Variables:</span>
<span class="c1">;; eval: (put 'insert 'racket-indent-function 'defun)</span>
<span class="c1">;; eval: (put 'update 'racket-indent-function 'defun)</span>
<span class="c1">;; eval: (put 'delete 'racket-indent-function 'defun)</span>
<span class="c1">;; eval: (put 'select 'racket-indent-function 'defun)</span>
<span class="c1">;; End:</span>
</pre></div>
</td></tr></tbody></table>
</div>
<p>Although this works, it’s not ideal.</p>
<p>So it would be nice for racket-mode to use a <code>#lang</code>-supplied <a href="https://docs.racket-lang.org/tools/lang-languages-customization.html#(part._.Indentation)"><code>drracket:indentation</code></a>.</p>
<p>However, even that doesn’t seem enough: Not just a full-blown <code>#lang</code>, but really <em>any</em> library module — like <code>sql</code> — ought to be able to specify indent for special syntax.</p>
<p>So I don’t yet know a good story for rich indentation that is both complete and performant.</p>
<h1 id="conclusion">Conclusion</h1>
<p>Hopefully this gives you some overview of how racket-mode works, as well as what it currently does or does not attempt to do, and why.</p>
<div class="footnotes">
<ol>
<li id="racket-mode-footnote-1-definition" class="footnote-definition">
<p>Another approach would be to split it into an Emacs package and a Racket package. This would have various pros and cons. I think the main disadvantage is that users would need both to update to stay in sync. When using CIDER for Clojure I found this a little troublesome. <a href="#racket-mode-footnote-1-return">↩</a></p></li>
<li id="racket-mode-footnote-2-definition" class="footnote-definition">
<p>Since <code>,</code> is reader shorthand for <code>unquote</code>, which is only valid inside a <code>quasiquote</code>, this is a clever way to distinguish a command from a valid Racket expression. <a href="#racket-mode-footnote-2-return">↩</a></p></li>
<li id="racket-mode-footnote-3-definition" class="footnote-definition">
<p>The nonce needn’t be a cryptographically secure random number, just unique; an increasing integer is fine. <a href="#racket-mode-footnote-3-return">↩</a></p></li>
<li id="racket-mode-footnote-4-definition" class="footnote-definition">
<p>“Point” is Emacs’ name for what you might call the cursor or caret. <a href="#racket-mode-footnote-4-return">↩</a></p></li>
<li id="racket-mode-footnote-5-definition" class="footnote-definition">
<p>As far as I can tell, the convention is that function applications should be indented consistently — only macro invocations might use custom indent. <a href="#racket-mode-footnote-5-return">↩</a></p></li></ol></div>
<footer></footer></article></div>Extramaze LLC: Using system fonts (not Google fonts)urn:www-greghendershott-com:-2018-05-extramaze-llc-using-system-fonts-not-google-fonts.html2018-05-20T04:00:00Z2018-05-20T04:00:00ZGreg Hendershott
<div>
<article>
<header>
<h1>Extramaze LLC: Using system fonts (not Google fonts)</h1>
<p class="date-and-tags">
<time datetime="2018-05-20" pubdate="true">2018-05-20</time> :: <span class="tags"><a href="/tags/Racket.html">Racket</a>, <a href="/tags/Extramaze.html">Extramaze</a></span></p></header>
<blockquote>
<p>Update: Due to lack of interest/use, in June 2021 this site was shut down and user data (emails, names, search alerts) deleted from all systems and backups.</p></blockquote>
<p>In my <a href="/2018/05/extramaze-llc-using-racket-postgresql-aws-but-no-ads-or-js.html">previous post</a> I discussed what I’m doing with <a href="https://deals.extramaze.com">deals.extramaze.com</a> — and what I’m intentionally <em>not</em> doing. Since then, I’m not-doing more. This improves performance and simplifies the content security policy.</p><!-- more-->
<p>In the <a href="/2018/05/extramaze-llc-using-racket-postgresql-aws-but-no-ads-or-js.html#trying-to-do-the-right-thing-privacy">privacy</a> section I described various third-party services the site is <em>not</em> using. On <a href="https://www.recurse.com/">Recurse Center’s</a> private chat (<a href="https://zulipchat.com">Zulip</a>), <a href="https://twitter.com/jab_______">Josh Bronson</a> pointed out that using Google-hosted fonts might not be the best idea:</p>
<ul>
<li>
<p>Privacy: Using <code>fonts.googleapis.com</code> tells Google what sites the user is visiting. It doesn’t make much sense to avoid Google Analytics but keep using Google Fonts.</p></li>
<li>
<p>Performance: Downloading custom fonts takes time. This can be especially annoying on slower mobile connections.</p></li></ul>
<p>Often these are two sides of the same coin: Leaking information to a third party usually consumes extra time and bandwidth to make extra connections and transfer data.</p>
<p>Instead host the font files locally? That would address privacy. But it could make performance worse not better. Using fonts hosted by Google <em>does</em> have the advantage that the user might already have downloaded them, as a result of some other web site using them.</p>
<p>Josh asked if I’d considered using system fonts, and shared <a href="https://furbo.org/2018/03/28/system-fonts-in-css/">two</a> <a href="http://markdotto.com/2018/02/07/github-system-fonts/">links</a>. Initially I was reluctant — but but but… my carefully chosen fonts! And yes, it changed the look and feel of the site, somewhat. But after living with it for about half an hour, I thought it was just fine.</p>
<h1 id="accessibility">Accessibility</h1>
<p>At the same time I was looking at fonts, I’d been starting to review accessibility issues. Such as making sure that:</p>
<ul>
<li><code>img</code> elements have <code>alt</code> attributes</li>
<li><code>svg</code> elements have child <code>title</code> elements</li>
<li>the <code>html</code> element has a <code>lang=en</code> attribute</li>
<li>all <code>id</code> attributes on a page are unique</li></ul>
<p>Another item on this list: Ensuring that font colors have sufficient contrast. So I did that while I had the hood open.</p>
<p>(I have more work to do for accessibility: I’ve only just started to use the site with the macOS screen reader. Maybe this should be another blog post.)</p>
<h1 id="content-security-policy">Content Security Policy</h1>
<p>Dropping Google hosted fonts let me simplify the content security policy. Indeed it seems to have cut down on the number of violation reports.</p>
<p>Since my last blog post, I had temporarily switched from <code>Content-Security-Policy</code> to <code>Content-Security-Policy-Report-Only</code>. I was nervous because I didn’t understand all the violations. <a href="https://report-uri.com">report-uri.com</a> helps by filtering things like violations caused by browser extensions. But even after such filtering, I had violations that made no sense to me.</p>
<p>Using system fonts, that situation improved significantly. I feel good about cranking it back up again to be enforced instead of report-only.<sup><a href="#extramaze-llc-using-system-fonts-not-google-fonts-footnote-1-definition" name="extramaze-llc-using-system-fonts-not-google-fonts-footnote-1-return">1</a></sup></p>
<p>I suppose this illustrates a concern using any third-party service. Effectively it mutates your web site with various scripts or fonts or styles. What are all its mutations? You may think you know from observation: Oh it needs a script from a certain URI. But oops, in a certain scenario it turns out that it also needs a style from another URI. Surprise. And that’s just today. What if the service changes in the future?</p>
<p>Really, every service ought to state exactly what content security policy settings it needs to work.</p>
<div class="footnotes">
<ol>
<li id="extramaze-llc-using-system-fonts-not-google-fonts-footnote-1-definition" class="footnote-definition">
<p>Of course you can use both headers. <code>Content-Security-Policy</code> is what you’re enforcing, and <code>Content-Security-Policy-Report-Only</code> can be used to dry-run changes. <a href="#extramaze-llc-using-system-fonts-not-google-fonts-footnote-1-return">↩</a></p></li></ol></div>
<footer></footer></article></div>Extramaze LLC: Using Racket, PostgreSQL, AWS (but no ads or JS)urn:www-greghendershott-com:-2018-05-extramaze-llc-using-racket-postgresql-aws-but-no-ads-or-js.html2018-05-04T04:00:00Z2018-05-04T04:00:00ZGreg Hendershott
<div>
<article>
<header>
<h1>Extramaze LLC: Using Racket, PostgreSQL, AWS (but no ads or JS)</h1>
<p class="date-and-tags">
<time datetime="2018-05-04" pubdate="true">2018-05-04</time> :: <span class="tags"><a href="/tags/Racket.html">Racket</a>, <a href="/tags/Extramaze.html">Extramaze</a></span></p></header>
<blockquote>
<p>Update: Due to lack of interest/use, in June 2021 this site was shut down and user data (emails, names, search alerts) deleted from all systems and backups.</p></blockquote>
<p>For Extramaze LLC I’m using Racket in a commercial project — a search engine with email alerts for deals on music gear — <a href="https://deals.extramaze.com">deals.extramaze.com</a>.</p>
<p>This blog post is a sort of whirlwind tour of the use case and business model, as well as how I use things like Racket, PostgreSQL, and AWS — but don’t use advertising or JavaScript.</p><!-- more-->
<h1 id="use-case">Use case</h1>
<p>Who needs this?</p>
<p>Let’s say I play guitar. In fact, I seem to collect guitars. My partner is skeptical of this use of financial and space resources. I have my eye on a certain 7-string guitar. I don’t <em>need</em> it. But I want it. If it goes on sale in the months ahead, I’d like to grab it. I can point out that I saved (say) $500. Although my partner will see right through this pathetic justification, they will appreciate that at least I am hearing them and making an effort. I hope.</p>
<p>Or more virtuously: I’m a parent and my child needs a trombone when school starts this autumn. I create an alert in case something goes on sale over the summer.</p>
<h1 id="business-model">Business model</h1>
<p>Currently you must join (create an account) to use the site. It’s free to search deals that are at least a day old. If you subscribe (pay), you can search the very newest deals — and save searches to get alerted when matching deals appear.</p>
<p>Why ask people to pay? I’m disenchanted with services that survive on money from advertising — where users are the product being sold, and the actual customers are advertisers.</p>
<p>Instead I want to do something where <strong>users = customers</strong>.</p>
<ul>
<li>
<p>I like making things that people like enough to pay for.</p></li>
<li>
<p>As a practical matter, it can be hard enough to make one constituency happy. Two is double the difficulty — even more when interests conflict. A small company has enough challenges; why build more into the foundation.</p></li></ul>
<p>If it turns out that not enough people want to pay for this service? I’d rather discontinue it than turn to advertising.</p>
<h1 id="picking-a-name">Picking a name</h1>
<p>The last time I picked a company name, the name was “Cakewalk”<sup><a href="#extramaze-llc-using-racket-postgresql-aws-but-no-ads-or-js-footnote-1-definition" name="extramaze-llc-using-racket-postgresql-aws-but-no-ads-or-js-footnote-1-return">1</a></sup> and the year was 1987. Nearly a decade passed, the internet grew popular, and we registered <code>cakewalk.com</code>. Easy.</p>
<p>The experience now is… different. You may be surprised to learn that many desirable domain names are already registered. I know, right? After too many days agonizing with dictionaries and a thesaurus, I settled on a contraction of “extra” and “amaze” — Extramaze.</p>
<p>It turns out that “extramaze” is sometimes used to describe cues outside a lab maze. Although that’s a bit creepy, I’m relieved the cues are <em>outside</em> the maze — this alludes to beacons of hope shining on those of us stuck inside our own mazes! Or something. Moving on….</p>
<h1 id="trying-to-do-the-right-thing-privacy">Trying to do the right thing: Privacy</h1>
<p>The web site does not use third-party JavaScript. No Google Analytics. No social buttons. Nothing.<sup><a href="#extramaze-llc-using-racket-postgresql-aws-but-no-ads-or-js-footnote-2-definition" name="extramaze-llc-using-racket-postgresql-aws-but-no-ads-or-js-footnote-2-return">2</a></sup> The one exception — if you subscribe — is Stripe for payments.</p>
<p>If you don’t believe me look at the relevant parts of the <code>Content-Security-Policy</code> response header value:</p>
<pre><code>default-src 'none'; script-src https://checkout.stripe.com;</code></pre>
<p>(Indeed notice that <code>script-src</code> doesn’t include <code>'self'</code> — the site itself supplies no JavaScript. If you ask “Why not?” I can only respond with another question: “Why?”. The UX doesn’t require it. Maybe someday. Meanwhile it is one less facet to develop, test, debug, and examine for vulnerabilities.)</p>
<p>Recently I experimented with Google and Twitter ads. Click-throughs were OK but conversion was poor. I wondered if account-creation was a speed bump, especially on mobile. So I added “Login with {Google, Twitter}” buttons. Of course if you use those, you are sharing some information with {Google, Twitter}. However you can still create and use a native Extramaze account.</p>
<p>So much for third-party data collection. How about first-party?</p>
<ul>
<li>The web site does basic web server request and response logging.</li>
<li>When people create an account:
<ul>
<li>The service asks for:
<ul>
<li>an email address</li>
<li>a how-would-you-like-to-be-addressed name</li>
<li>a password</li></ul></li>
<li>Successful and failed logins are stored — the latter to do an exponential back-off on subsequent attempts.</li></ul></li>
<li>When people also pay to subscribe:
<ul>
<li>Stripe asks for credit card and zip code information. Extramaze does not see or store this.</li>
<li>They gain the ability to create saved searches, and get email alerts. Of course we need to store the search phrase, e.g. <code>7-string left hand guitar</code>.</li></ul></li></ul>
<p>Beyond that, the service collects no personal information.</p>
<h1 id="trying-to-do-the-right-thing-security">Trying to do the right thing: Security</h1>
<p>I’ve spent a lot of time thinking through the privacy and security implications of account sign-up and password resets. This was the scariest part for me. I found Troy Hunt’s <a href="https://www.troyhunt.com/everything-you-ever-wanted-to-know/"><em>Everything you ever wanted to know about building a secure password reset feature</em></a> incredibly helpful and follow its recommendations. Also useful: <a href="https://stackoverflow.com/questions/549/the-definitive-guide-to-form-based-website-authentication"><em>The definitive guide to form-based website authentication</em></a>.</p>
<p>I already mentioned <code>Content-Security-Policy</code>. Information about this and other security headers is at <a href="https://securityheaders.com/?q=https%3A%2F%2Fdeals.extramaze.com&followRedirects=on">securityheaders.com</a>. Some headers let browsers report problems and <a href="https://report-uri.com">https://report-uri.com</a> is helpful here.</p>
<p>I won’t say, “We take security very seriously”, because that’s become a cliché. (Is there a web site that does for incident disclosures what OurIncredibleJourney.com does for acquihire notices?)</p>
<p>But. I think about security frequently. I read as much as I can about other people’s experiences and recommendations. I assume that vulnerabilities exist and the only question is who will find them; I need to exercise a commercially reasonable effort to find them first.</p>
<h1 id="ingredients">Ingredients</h1>
<p>There are three main pieces:</p>
<ul>
<li>Crawl certain music retailer web sites looking for deals (daily).</li>
<li>Send search alert emails (daily).</li>
<li>Run a web site (24/7).</li></ul>
<p>All use some mix of:</p>
<ul>
<li>Racket</li>
<li>PostgreSQL</li>
<li>AWS (EC2, ELB, ECS, SES, SNS, RDS, CloudWatch)</li></ul>
<h2 id="racket-crawling-and-scraping">Racket: Crawling and scraping</h2>
<p>Most music product web stores have a section — variously called “outlet”, “clearance”, “deal zone”, or “blowout” — where the deals are featured. Deals include demo units, price drops, and so on. For demo units the product might be “like new” condition, or there might be a cosmetic flaw which is described.</p>
<p>Currently we crawl these areas of <a href="https://www.sweetwater.com/dealzone/">Sweetwater</a> and <a href="https://www.zzounds.com/blowouts">Zzounds</a>.</p>
<p>More would be better. On the other hand, one step at a time. Furthermore, these days many web sites block crawlers by default. It doesn’t matter if you <code>GET /robots.txt</code> and follow its rules scrupulously. Instead you must ask a human for permission and get white-listed. When you ask, the human might say no, or simply not reply at all.</p>
<p>Racket has great “batteries included” for making HTTP requests, and parsing HTML responses into <a href="https://docs.racket-lang.org/xml/index.html#%28def._%28%28lib._xml%2Fprivate%2Fxexpr-core..rkt%29._xexpr~3f%29%29">x-expressions</a>. That is, HTML such as</p>
<div class="brush: html">
<table class="sourcetable">
<tbody>
<tr>
<td class="linenos">
<div class="linenodiv">
<pre>1
2
3
4
5
6</pre></div></td>
<td class="code">
<div class="source">
<pre><span></span><span class="p"><</span><span class="nt">p</span> <span class="na">class</span><span class="o">=</span><span class="s">"class"</span><span class="p">></span>
Some
<span class="p"><</span><span class="nt">em</span><span class="p">></span>awesome<span class="p"></</span><span class="nt">em</span><span class="p">></span>
HTML
<span class="ni">&tm;</span>
<span class="p"></</span><span class="nt">p</span><span class="p">></span>
</pre></div>
</td></tr></tbody></table>
</div>
<p>is equivalent to the x-expression</p>
<div class="brush: racket">
<table class="sourcetable">
<tbody>
<tr>
<td class="linenos">
<div class="linenodiv">
<pre>1
2
3
4
5</pre></div></td>
<td class="code">
<div class="source">
<pre><span></span><span class="o">'</span><span class="p">(</span><span class="ss">p</span> <span class="p">([</span><span class="ss"><a href="http://docs.racket-lang.org/reference/createclass.html#(form._((lib._racket/private/class-internal..rkt)._class))" style="color: inherit">class</a></span> <span class="s2">"class"</span><span class="p">])</span>
<span class="s2">"Some "</span>
<span class="p">(</span><span class="ss">em</span> <span class="p">()</span> <span class="s2">"awesome"</span><span class="p">)</span>
<span class="s2">" HTML"</span>
<span class="ss">tm</span><span class="p">)</span>
</pre></div>
</td></tr></tbody></table>
</div>
<p>As the Lisp Evangelism Task Force will point out on Hacker News every few weeks, x-expressions are what XML would be if it weren’t produced by the Department of Redundancy Department.</p>
<p>Finding the “interesting bits” within an x-expression can be done in various ways. In simple cases, you can use Racket’s <code><a href="http://docs.racket-lang.org/reference/match.html#(form._((lib._racket/match..rkt)._match))" style="color: inherit">match</a></code>, but for complicated HTML that can be tedious and/or brittle. <code><a href="http://docs.racket-lang.org/xml/index.html#(def._((lib._xml/path..rkt)._se-path*/list))" style="color: inherit">se-path*/list</a></code> is a better idea, but you can only select a direct path of element tags. You can’t express CSS selector things like, “Find all <code>p</code> elements somewhere under <code>div</code>s having a <code>"container"</code> <code>class</code>”.</p>
<p>So I wrote a function to do a depth-first folding walk of an x-expression. In addition to accumulating a result value, it accumulates a “path” — a list of full x-expressions from “this” one up through its ancestors to the root, full x-expression:</p>
<div class="brush: racket">
<table class="sourcetable">
<tbody>
<tr>
<td class="linenos">
<div class="linenodiv">
<pre> 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17</pre></div></td>
<td class="code">
<div class="source">
<pre><span></span><span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/define.html#(form._((lib._racket/private/base..rkt)._define))" style="color: inherit">define</a></span> <span class="nb"><a href="http://docs.racket-lang.org/reference/Manipulating_Paths.html#(def._((quote._~23~25kernel)._path~3f))" style="color: inherit">path?</a></span> <span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/data-structure-contracts.html#(def._((lib._racket/contract/base..rkt)._listof))" style="color: inherit">listof</a></span> <span class="n"><a href="http://docs.racket-lang.org/xml/index.html#(def._((lib._xml/private/xexpr-core..rkt)._xexpr/c))" style="color: inherit">xexpr/c</a></span><span class="p">))</span>
<span class="c1">;; A depth-first folding walk of the xexpr. The "path" is a list of</span>
<span class="c1">;; xexpr from the current one to its ancestors. You can `match` on</span>
<span class="c1">;; this to do the equivalent of `se-path*/list`, but with the full</span>
<span class="c1">;; power of `match`.</span>
<span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/define.html#(form._((lib._racket/private/base..rkt)._define))" style="color: inherit">define</a></span> <span class="p">(</span><span class="n">walk</span> <span class="n">f</span> <span class="n">v</span> <span class="n">x</span><span class="p">)</span>
<span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/let.html#(form._((lib._racket/private/letstx-scheme..rkt)._let))" style="color: inherit">let</a></span> <span class="n">recur</span> <span class="p">([</span><span class="n">path</span> <span class="o">'</span><span class="p">()]</span>
<span class="p">[</span><span class="n">v</span> <span class="n">v</span><span class="p">]</span>
<span class="p">[</span><span class="n">x</span> <span class="n">x</span><span class="p">])</span>
<span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/define.html#(form._((lib._racket/private/base..rkt)._define))" style="color: inherit">define</a></span> <span class="n">this-path</span> <span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/pairs.html#(def._((quote._~23~25kernel)._cons))" style="color: inherit">cons</a></span> <span class="n">x</span> <span class="n">path</span><span class="p">))</span>
<span class="p">(</span><span class="n">f</span> <span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/match.html#(form._((lib._racket/match..rkt)._match))" style="color: inherit">match</a></span> <span class="n">x</span>
<span class="p">[(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/pairs.html#(def._((quote._~23~25kernel)._list*))" style="color: inherit">list*</a></span> <span class="p">(</span><span class="n">?</span> <span class="nb"><a href="http://docs.racket-lang.org/reference/symbols.html#(def._((quote._~23~25kernel)._symbol~3f))" style="color: inherit">symbol?</a></span> <span class="k"><a href="http://docs.racket-lang.org/reference/creatingunits.html#(form._((lib._racket/unit..rkt)._tag))" style="color: inherit">tag</a></span><span class="p">)</span> <span class="p">(</span><span class="n">?</span> <span class="nb"><a href="http://docs.racket-lang.org/reference/pairs.html#(def._((quote._~23~25kernel)._list~3f))" style="color: inherit">list?</a></span><span class="p">)</span> <span class="n">xs</span><span class="p">)</span>
<span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/for.html#(form._((lib._racket/private/base..rkt)._for/fold))" style="color: inherit">for/fold</a></span> <span class="p">([</span><span class="n">v</span> <span class="n">v</span><span class="p">])</span> <span class="p">([</span><span class="n">x</span> <span class="n">xs</span><span class="p">])</span>
<span class="p">(</span><span class="n">recur</span> <span class="n">this-path</span> <span class="n">v</span> <span class="n">x</span><span class="p">))]</span>
<span class="p">[</span><span class="k"><a href="http://docs.racket-lang.org/reference/stx-patterns.html#(form._((lib._racket/private/stxcase-scheme..rkt).__))" style="color: inherit">_</a></span> <span class="n">v</span><span class="p">])</span>
<span class="n">this-path</span><span class="p">)))</span>
</pre></div>
</td></tr></tbody></table>
</div>
<p>I wrapped this in a simple <code>select</code> function that <code>conjoin</code>s one or more predicates:</p>
<div class="brush: racket">
<table class="sourcetable">
<tbody>
<tr>
<td class="linenos">
<div class="linenodiv">
<pre> 1
2
3
4
5
6
7
8
9
10
11
12</pre></div></td>
<td class="code">
<div class="source">
<pre><span></span><span class="c1">;; Using quasiquoted match expressions with `walk` can be tedious and</span>
<span class="c1">;; error-prone. Often you end up specifying the match in more detail</span>
<span class="c1">;; than is really necessary. A sometimes friendlier way is to use</span>
<span class="c1">;; `select` with selector combinators grouped using `conjoin` (and</span>
<span class="c1">;; maybe `disjoin`).</span>
<span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/define.html#(form._((lib._racket/private/base..rkt)._define))" style="color: inherit">define</a></span> <span class="p">(</span><span class="n">select</span> <span class="n">x</span> <span class="o">.</span> <span class="n">fs</span><span class="p">)</span> <span class="c1">;(-> xexpr/c (-> path? any) ...+ list?)</span>
<span class="p">(</span><span class="n">walk</span> <span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/lambda.html#(form._((lib._racket/private/base..rkt)._~ce~bb))" style="color: inherit">λ</a></span> <span class="p">(</span><span class="n">vs</span> <span class="n">path</span><span class="p">)</span>
<span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/match.html#(form._((lib._racket/match..rkt)._match))" style="color: inherit">match</a></span> <span class="p">((</span><span class="nb"><a href="http://docs.racket-lang.org/reference/procedures.html#(def._((lib._racket/private/base..rkt)._apply))" style="color: inherit">apply</a></span> <span class="nb"><a href="http://docs.racket-lang.org/reference/procedures.html#(def._((lib._racket/function..rkt)._conjoin))" style="color: inherit">conjoin</a></span> <span class="n">fs</span><span class="p">)</span> <span class="n">path</span><span class="p">)</span>
<span class="p">[</span><span class="no">#f</span> <span class="n">vs</span><span class="p">]</span>
<span class="p">[</span><span class="n">v</span> <span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/pairs.html#(def._((quote._~23~25kernel)._cons))" style="color: inherit">cons</a></span> <span class="n">v</span> <span class="n">vs</span><span class="p">)]))</span>
<span class="o">'</span><span class="p">()</span>
<span class="n">x</span><span class="p">))</span>
</pre></div>
</td></tr></tbody></table>
</div>
<p>I wrote a few functions intended to be used as combinators with <code>conjoin</code>, to do roughly CSS selector style matching. A few example pieces:</p>
<div class="brush: racket">
<table class="sourcetable">
<tbody>
<tr>
<td class="linenos">
<div class="linenodiv">
<pre> 1
2
3
4
5
6
7
8
9
10
11
12</pre></div></td>
<td class="code">
<div class="source">
<pre><span></span><span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/define.html#(form._((lib._racket/private/base..rkt)._define))" style="color: inherit">define</a></span> <span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/createclass.html#(form._((lib._racket/private/class-internal..rkt)._this))" style="color: inherit">this</a></span> <span class="n">path</span><span class="p">)</span>
<span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/match.html#(form._((lib._racket/match..rkt)._match))" style="color: inherit">match</a></span> <span class="n">path</span>
<span class="p">[(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/pairs.html#(def._((quote._~23~25kernel)._cons))" style="color: inherit">cons</a></span> <span class="n">v</span> <span class="k"><a href="http://docs.racket-lang.org/reference/stx-patterns.html#(form._((lib._racket/private/stxcase-scheme..rkt).__))" style="color: inherit">_</a></span><span class="p">)</span> <span class="n">v</span><span class="p">]</span>
<span class="p">[</span><span class="k"><a href="http://docs.racket-lang.org/reference/stx-patterns.html#(form._((lib._racket/private/stxcase-scheme..rkt).__))" style="color: inherit">_</a></span> <span class="no">#f</span><span class="p">]))</span>
<span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/define.html#(form._((lib._racket/private/base..rkt)._define))" style="color: inherit">define</a></span> <span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/creatingunits.html#(form._((lib._racket/unit..rkt)._tag))" style="color: inherit">tag</a></span> <span class="n">path</span><span class="p">)</span>
<span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/match.html#(form._((lib._racket/match..rkt)._match))" style="color: inherit">match</a></span> <span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/createclass.html#(form._((lib._racket/private/class-internal..rkt)._this))" style="color: inherit">this</a></span> <span class="n">path</span><span class="p">)</span>
<span class="p">[(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/pairs.html#(def._((quote._~23~25kernel)._list))" style="color: inherit">list</a></span> <span class="p">(</span><span class="n">?</span> <span class="nb"><a href="http://docs.racket-lang.org/reference/symbols.html#(def._((quote._~23~25kernel)._symbol~3f))" style="color: inherit">symbol?</a></span> <span class="n">v</span><span class="p">)</span> <span class="k"><a href="http://docs.racket-lang.org/reference/stx-patterns.html#(form._((lib._racket/private/stxcase-scheme..rkt).__))" style="color: inherit">_</a></span> <span class="k"><a href="http://docs.racket-lang.org/reference/stx-patterns.html#(form._((lib._racket/private/stxcase-scheme..rkt)._......))" style="color: inherit">...</a></span><span class="p">)</span> <span class="n">v</span><span class="p">]</span>
<span class="p">[</span><span class="k"><a href="http://docs.racket-lang.org/reference/stx-patterns.html#(form._((lib._racket/private/stxcase-scheme..rkt).__))" style="color: inherit">_</a></span> <span class="no">#f</span><span class="p">]))</span>
<span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/define.html#(form._((lib._racket/private/base..rkt)._define))" style="color: inherit">define</a></span> <span class="p">((</span><span class="n">tag?</span> <span class="n">sym</span><span class="p">)</span> <span class="n">path</span><span class="p">)</span>
<span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/Equality.html#(def._((quote._~23~25kernel)._equal~3f))" style="color: inherit">equal?</a></span> <span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/creatingunits.html#(form._((lib._racket/unit..rkt)._tag))" style="color: inherit">tag</a></span> <span class="n">path</span><span class="p">)</span> <span class="n">sym</span><span class="p">))</span>
</pre></div>
</td></tr></tbody></table>
</div>
<p>For example, to extract the value of the <code>href</code> attribute for all <code>a.some-class</code> elements whose immediate parent is a <code>div.box-class</code>:</p>
<div class="brush: racket">
<table class="sourcetable">
<tbody>
<tr>
<td class="linenos">
<div class="linenodiv">
<pre>1
2
3
4
5
6</pre></div></td>
<td class="code">
<div class="source">
<pre><span></span><span class="p">(</span><span class="n">select</span> <span class="n">xexpr</span>
<span class="p">(</span><span class="n">tag?</span> <span class="o">'</span><span class="ss">a</span><span class="p">)</span>
<span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/objectutils.html#(def._((lib._racket/private/class-internal..rkt)._class~3f))" style="color: inherit">class?</a></span> <span class="s2">"some-class"</span><span class="p">)</span>
<span class="p">(</span><span class="n">parent?</span> <span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/procedures.html#(def._((lib._racket/function..rkt)._conjoin))" style="color: inherit">conjoin</a></span> <span class="p">(</span><span class="n">tag?</span> <span class="o">'</span><span class="ss">div</span><span class="p">)</span>
<span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/objectutils.html#(def._((lib._racket/private/class-internal..rkt)._class~3f))" style="color: inherit">class?</a></span> <span class="s2">"some-box"</span><span class="p">)))</span>
<span class="p">(</span><span class="n">attr-val</span> <span class="o">'</span><span class="ss">href</span><span class="p">))</span>
</pre></div>
</td></tr></tbody></table>
</div>
<p>Of course you can layer on some shorthand compositions like <code>tag.class?</code>:</p>
<div class="brush: racket">
<table class="sourcetable">
<tbody>
<tr>
<td class="linenos">
<div class="linenodiv">
<pre>1
2
3</pre></div></td>
<td class="code">
<div class="source">
<pre><span></span><span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/define.html#(form._((lib._racket/private/base..rkt)._define))" style="color: inherit">define</a></span> <span class="p">(</span><span class="n">tag.class?</span> <span class="n">sym</span> <span class="k"><a href="http://docs.racket-lang.org/reference/createclass.html#(form._((lib._racket/private/class-internal..rkt)._class))" style="color: inherit">class</a></span><span class="p">)</span>
<span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/procedures.html#(def._((lib._racket/function..rkt)._conjoin))" style="color: inherit">conjoin</a></span> <span class="p">(</span><span class="n">tag?</span> <span class="n">sym</span><span class="p">)</span>
<span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/objectutils.html#(def._((lib._racket/private/class-internal..rkt)._class~3f))" style="color: inherit">class?</a></span> <span class="k"><a href="http://docs.racket-lang.org/reference/createclass.html#(form._((lib._racket/private/class-internal..rkt)._class))" style="color: inherit">class</a></span><span class="p">)))</span>
</pre></div>
</td></tr></tbody></table>
</div>
<p>And the example becomes:</p>
<div class="brush: racket">
<table class="sourcetable">
<tbody>
<tr>
<td class="linenos">
<div class="linenodiv">
<pre>1
2
3
4</pre></div></td>
<td class="code">
<div class="source">
<pre><span></span><span class="p">(</span><span class="n">select</span> <span class="n">xexpr</span>
<span class="p">(</span><span class="n">tag.class?</span> <span class="o">'</span><span class="ss">a</span> <span class="s2">"some-class"</span><span class="p">)</span>
<span class="p">(</span><span class="n">parent?</span> <span class="p">(</span><span class="n">tag.class?</span> <span class="o">'</span><span class="ss">div</span> <span class="s2">"some-box"</span><span class="p">)</span>
<span class="p">(</span><span class="n">attr-val</span> <span class="o">'</span><span class="ss">href</span><span class="p">))</span>
</pre></div>
</td></tr></tbody></table>
</div>
<p>You <em>could</em> also write a parser from CSS selector syntax to these <code>select</code> expressions — maybe even define a <code>#lang css-selector</code>. But I like s-expressions. Even if I didn’t, I’m not using this extensively enough to warrant that work.</p>
<p>Likewise, although I’d like to open-source this as a distinct, complete project, I just haven’t had time to review it thoroughly, write documentation, and so on.</p>
<h2 id="racket-web-server">Racket: web server</h2>
<p>Racket has great “batteries included” for making web site servers. In addition to Racket’s documentation for this, I can recommend Jesse Alama’s resources including his blog <a href="https://lisp.sh">lisp.sh</a> and his book <a href="https://serverracket.com/">Server: Racket</a>.</p>
<p>Racket provides a macro called <a href="https://docs.racket-lang.org/web-server/dispatch.html"><code>dispatch-rules</code></a> to define bi-directional “routes”. You give it rules, of each which is a URI pattern and a handler function. It defines two functions covering all of your rules:</p>
<ul>
<li>A <code>request? -> response?</code> dispatcher.</li>
<li>A <code>procedure? -> string?</code> URI path maker.</li></ul>
<div class="brush: racket">
<table class="sourcetable">
<tbody>
<tr>
<td class="linenos">
<div class="linenodiv">
<pre>1
2
3
4
5
6
7</pre></div></td>
<td class="code">
<div class="source">
<pre><span></span><span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/define.html#(form._((quote._~23~25kernel)._define-values))" style="color: inherit">define-values</a></span> <span class="p">(</span><span class="n">dispatch</span> <span class="n">handler->path</span><span class="p">)</span>
<span class="p">(</span><span class="n">dispatch-rules</span>
<span class="p">[(</span><span class="s2">""</span><span class="p">)</span> <span class="n">home</span><span class="p">]</span>
<span class="p">[(</span><span class="s2">"about"</span><span class="p">)</span> <span class="n">about</span><span class="p">]</span>
<span class="p">[(</span><span class="s2">"path"</span> <span class="s2">"to"</span> <span class="s2">"foo"</span><span class="p">)</span> <span class="n">path/to/foo</span><span class="p">]</span>
<span class="p">[(</span><span class="s2">"user"</span> <span class="p">(</span><span class="n">string-arg</span><span class="p">))</span> <span class="n">user/id</span><span class="p">]</span>
<span class="p">[</span><span class="k"><a href="http://docs.racket-lang.org/reference/if.html#(form._((lib._racket/private/letstx-scheme..rkt)._else))" style="color: inherit">else</a></span> <span class="n">not-found</span><span class="p">]))</span>
</pre></div>
</td></tr></tbody></table>
</div>
<p>Because I needed to do authorization, I wrapped this in my own macro, <code>dispatch-rules+roles</code>. This also defines a third function, <code>request?
-> roles?</code>: Given a request that matches one of the patterns, what roles is a user required to have to be authorized to access it?</p>
<p>In the following example, we use three roles:</p>
<ul>
<li>
<p><code>'anon</code> means an anonymous user.</p></li>
<li>
<p><code>'free</code> is a user who is authenticated (logged in); certain routes should only be available to them.</p></li>
<li>
<p><code>'paid</code> is an authenticated user who has also subscribed (paid) and therefore is authorized for even more routes.</p></li></ul>
<div class="brush: racket">
<table class="sourcetable">
<tbody>
<tr>
<td class="linenos">
<div class="linenodiv">
<pre> 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17</pre></div></td>
<td class="code">
<div class="source">
<pre><span></span><span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/define.html#(form._((quote._~23~25kernel)._define-values))" style="color: inherit">define-values</a></span> <span class="p">(</span><span class="n">dispatch</span> <span class="n">handler->path</span> <span class="n">request->roles</span><span class="p">)</span>
<span class="p">(</span><span class="n">dispatch-rules+roles</span>
<span class="c1">;; Routes requiring 'anon or 'free or 'paid roles</span>
<span class="p">[(</span><span class="n">anon</span> <span class="n"><a href="http://docs.racket-lang.org/foreign/foreign_pointer-funcs.html#(def._((quote._~23~25foreign)._free))" style="color: inherit">free</a></span> <span class="n">paid</span><span class="p">)</span>
<span class="p">[(</span><span class="s2">""</span><span class="p">)</span> <span class="n">home</span><span class="p">]</span>
<span class="p">[(</span><span class="s2">"about"</span><span class="p">)</span> <span class="n">about</span><span class="p">]</span>
<span class="p">[(</span><span class="s2">"join"</span><span class="p">)</span> <span class="n">join</span><span class="p">]]</span>
<span class="c1">;; Routes requiring 'free or 'paid roles</span>
<span class="p">[(</span><span class="n"><a href="http://docs.racket-lang.org/foreign/foreign_pointer-funcs.html#(def._((quote._~23~25foreign)._free))" style="color: inherit">free</a></span> <span class="n">paid</span><span class="p">)</span>
<span class="p">[(</span><span class="n">logout</span><span class="p">)</span> <span class="n">logout</span><span class="p">]</span>
<span class="p">[(</span><span class="n">preferences</span><span class="p">)</span> <span class="n">preferences</span><span class="p">]</span>
<span class="p">[(</span><span class="n">subscribe</span><span class="p">)</span> <span class="n">subscribe</span><span class="p">]</span>
<span class="c1">;; Routes requiring 'paid role</span>
<span class="p">[(</span><span class="n">paid</span><span class="p">)</span>
<span class="p">[(</span><span class="n">payment</span><span class="p">)</span> <span class="n">payment</span><span class="p">]</span>
<span class="p">[(</span><span class="n">unsubscribe</span><span class="p">)</span> <span class="n">unsubscribe</span><span class="p">]]</span>
<span class="p">[</span><span class="k"><a href="http://docs.racket-lang.org/reference/if.html#(form._((lib._racket/private/letstx-scheme..rkt)._else))" style="color: inherit">else</a></span> <span class="n">not-found</span><span class="p">]))</span>
</pre></div>
</td></tr></tbody></table>
</div>
<p>My <code>dispatch-rules+roles</code> macro doesn’t itself do authorization — it defines the same old <code>dispatch</code> function that plain <code>dispatch-rules</code> does. You need to wrap that <code>dispatch</code> with something that consults <code>request->roles</code> and calls <code>dispatch</code> — or returns a <code>403 Forbidden</code> response (for an API) or redirects to a login or subscribe page (web app).</p>
<p>Of course, <em>that</em> in turn should be wrapped in something that sets the current user (so we know their roles) from e.g. a session key — or returns a <code>401</code> response (for an API) or redirects to a login page (web app).</p>
<p>Speaking of multiple wrappers around <code>dispatch</code>, this is a nice way to compose functionality, which I’ve seen in the Clojure <a href="https://github.com/ring-clojure/ring/wiki/Concepts#middleware">Ring</a> community. It’s cleaner to have one wrapper per bit of functionality, as opposed to one handler with a monolithic hairball of conditionals.</p>
<p>So a <code>handler?</code> is a function from <code>request?</code> to <code>response?</code>, like <code>dispatch</code>. A <code>wrapper?</code> is a function that takes a <code>handler?</code>, and returns a new <code>handler?</code>.</p>
<div class="brush: racket">
<table class="sourcetable">
<tbody>
<tr>
<td class="linenos">
<div class="linenodiv">
<pre>1
2</pre></div></td>
<td class="code">
<div class="source">
<pre><span></span><span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/define.html#(form._((lib._racket/private/base..rkt)._define))" style="color: inherit">define</a></span> <span class="n">handler?</span> <span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/function-contracts.html#(form._((lib._racket/contract/base..rkt)._-~3e))" style="color: inherit">-></a></span> <span class="n">request?</span> <span class="n">response?</span><span class="p">))</span>
<span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/define.html#(form._((lib._racket/private/base..rkt)._define))" style="color: inherit">define</a></span> <span class="n">wrapper?</span> <span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/function-contracts.html#(form._((lib._racket/contract/base..rkt)._-~3e))" style="color: inherit">-></a></span> <span class="n">handler?</span> <span class="n">handler?</span><span class="p">))</span>
</pre></div>
</td></tr></tbody></table>
</div>
<p>For instance, a wrapper to enforce https:</p>
<div class="brush: racket">
<table class="sourcetable">
<tbody>
<tr>
<td class="linenos">
<div class="linenodiv">
<pre> 1
2
3
4
5
6
7
8
9
10
11
12
13
14</pre></div></td>
<td class="code">
<div class="source">
<pre><span></span><span class="c1">;; This assumes we're behind ELB or nginx which gets both http and</span>
<span class="c1">;; https, talks to us only via http, setting an X-Forwarded-Proto</span>
<span class="c1">;; header to say the original protocol and an X-Forwarded-For header</span>
<span class="c1">;; to say the original IP.</span>
<span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/attaching-contracts-to-values.html#(form._((lib._racket/contract/region..rkt)._define/contract))" style="color: inherit">define/contract</a></span> <span class="p">((</span><span class="n">wrap-http->https</span> <span class="n">handler</span><span class="p">)</span> <span class="n">req</span><span class="p">)</span> <span class="n">wrapper?</span>
<span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/match.html#(form._((lib._racket/match..rkt)._match))" style="color: inherit">match</a></span> <span class="p">(</span><span class="n">headers-assq*</span> <span class="s2">#"x-forwarded-proto"</span> <span class="p">(</span><span class="n">request-headers/raw</span> <span class="n">req</span><span class="p">))</span>
<span class="p">[(</span><span class="n">header</span> <span class="k"><a href="http://docs.racket-lang.org/reference/stx-patterns.html#(form._((lib._racket/private/stxcase-scheme..rkt).__))" style="color: inherit">_</a></span> <span class="s2">#"http"</span><span class="p">)</span>
<span class="p">(</span><span class="n">redirect-to</span> <span class="p">(</span><span class="n">path->external-uri</span>
<span class="p">(</span><span class="n"><a href="http://docs.racket-lang.org/net/url.html#(def._((lib._net/url..rkt)._url-~3estring))" style="color: inherit">url->string</a></span> <span class="p">(</span><span class="k"><a href="http://docs.racket-lang.org/reference/struct-copy.html#(form._((lib._racket/private/base..rkt)._struct-copy))" style="color: inherit">struct-copy</a></span> <span class="n">url</span> <span class="p">(</span><span class="n">request-uri</span> <span class="n">req</span><span class="p">)</span>
<span class="p">[</span><span class="n">scheme</span> <span class="no">#f</span><span class="p">]</span>
<span class="p">[</span><span class="n">port</span> <span class="no">#f</span><span class="p">])))</span>
<span class="n">permanently</span><span class="p">)]</span>
<span class="p">[</span><span class="k"><a href="http://docs.racket-lang.org/reference/stx-patterns.html#(form._((lib._racket/private/stxcase-scheme..rkt).__))" style="color: inherit">_</a></span> <span class="p">(</span><span class="n">handler</span> <span class="n">req</span><span class="p">)]))</span>
</pre></div>
</td></tr></tbody></table>
</div>
<p>A whole chain of such wrappers can be composed — using <code>compose</code> or the <code>~></code> threading macro — to wrap the original <code>dispatch</code> function when we start the Racket web server:</p>
<div class="brush: racket">
<table class="sourcetable">
<tbody>
<tr>
<td class="linenos">
<div class="linenodiv">
<pre> 1
2
3
4
5
6
7
8
9
10
11
12
13
14</pre></div></td>
<td class="code">
<div class="source">
<pre><span></span><span class="p">(</span><span class="n">serve/servlet</span> <span class="p">(</span><span class="n">~></span> <span class="c1">;Note: requests go UP this chain, responses DOWN</span>
<span class="n">dispatch</span>
<span class="n">wrap-gzip</span>
<span class="n">wrap-not-modified</span>
<span class="n">wrap-authorize</span>
<span class="n">wrap-authenticate</span>
<span class="n">wrap-http->https</span>
<span class="n">wrap-timed-and-logged</span><span class="p">)</span>
<span class="kd">#:servlet-path</span> <span class="s2">"/"</span>
<span class="kd">#:servlet-regexp</span> <span class="sr">#px""</span>
<span class="kd">#:listen-ip</span> <span class="no">#f</span>
<span class="kd">#:port</span> <span class="p">(</span><span class="n">current-internal-port</span><span class="p">)</span>
<span class="kd">#:launch-browser?</span> <span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/booleans.html#(def._((quote._~23~25kernel)._not))" style="color: inherit">not</a></span> <span class="p">(</span><span class="n">current-production</span><span class="p">))</span>
<span class="kd">#:servlet-responder</span> <span class="n">error-responder</span><span class="p">)</span>
</pre></div>
</td></tr></tbody></table>
</div>
<h1 id="postgresql">PostgreSQL</h1>
<p>I was skeptical about using PostgreSQL: Is it web scale?</p>
<p>Seriously, I don’t have anything very exciting to describe about using PostgreSQL for this — which is wonderful. Currently:</p>
<ul>
<li>
<p>I’m hosting at AWS RDS.</p></li>
<li>
<p>Initial capture from crawling goes into a simple <a href="https://en.wikipedia.org/wiki/Star_schema">star schema</a>. The fact table has information about each deal, including its first-seen and last-seen times. When we already have a row for a deal, we just update the last-seen time. Dimension tables are what you’d expect — brand, product, and so on.</p></li>
<li>
<p>A simple view joins to denormalize, fitting the kind of query done by the web site’s search page. This is already quite fast; a materialized view makes it even faster.</p></li>
<li>
<p>Full-text search is delicious.</p></li></ul>
<p>Toward the end of my time at Cakewalk, I got some experience with Microsoft SQL Server, including optimizations. As a result, I’m aware that I can likewise do much more with PostgreSQL — looking at query execution plans, tuning indexes and queries, and so on. For now I’m satisfied I know roughly what I can do if/as/when necessary.</p>
<p>Racket has an excellent <a href="http://docs.racket-lang.org/db/index.html">db</a> package. It also has a <a href="http://docs.racket-lang.org/sql/index.html">sql</a> package that lets you write SQL as s-expressions rather than blobs of text, for example:</p>
<div class="brush: sql">
<table class="sourcetable">
<tbody>
<tr>
<td class="linenos">
<div class="linenodiv">
<pre>1
2
3</pre></div></td>
<td class="code">
<div class="source">
<pre><span></span><span class="k">SELECT</span> <span class="k">first</span><span class="p">,</span> <span class="k">last</span>
<span class="k">FROM</span> <span class="n">tribbles</span>
<span class="k">WHERE</span> <span class="n">id</span> <span class="o">=</span> <span class="err">$</span><span class="mi">1</span>
</pre></div>
</td></tr></tbody></table>
</div>
<div class="brush: racket">
<table class="sourcetable">
<tbody>
<tr>
<td class="linenos">
<div class="linenodiv">
<pre>1
2
3</pre></div></td>
<td class="code">
<div class="source">
<pre><span></span><span class="p">(</span><span class="n">select</span> <span class="nb"><a href="http://docs.racket-lang.org/reference/pairs.html#(def._((lib._racket/list..rkt)._first))" style="color: inherit">first</a></span> <span class="nb"><a href="http://docs.racket-lang.org/reference/pairs.html#(def._((lib._racket/list..rkt)._last))" style="color: inherit">last</a></span>
<span class="kd">#:from</span> <span class="n">tribbles</span>
<span class="kd">#:where</span> <span class="p">(</span><span class="nb"><a href="http://docs.racket-lang.org/reference/generic-numbers.html#(def._((quote._~23~25kernel)._~3d))" style="color: inherit">=</a></span> <span class="n"><a href="http://docs.racket-lang.org/syntax/Library_Syntax_Classes_and_Literal_Sets.html#(form._((lib._syntax/parse..rkt)._id))" style="color: inherit">id</a></span> <span class="o">,</span><span class="n"><a href="http://docs.racket-lang.org/syntax/Library_Syntax_Classes_and_Literal_Sets.html#(form._((lib._syntax/parse..rkt)._id))" style="color: inherit">id</a></span><span class="p">))</span>
</pre></div>
</td></tr></tbody></table>
</div>
<h1 id="aws">AWS</h1>
<p>I’m using Amazon Web Services because I feel badly that it is such an unpopular choice and want to see them get at least a <em>little</em> business.</p>
<p>Seriously, this has been a reasonable choice to get going quickly and it is affordable initially within the free tier.</p>
<p>One of my earliest Racket packages is <a href="https://pkgs.racket-lang.org/package/aws">aws</a>. Although I’ve used it intermittently, lately I’m <em>really</em> eating my own dog food.</p>
<p>I made a local change to support getting AWS credentials from EC2 instance meta-data. After living with that in production for a couple months, I shared that back in <a href="https://github.com/greghendershott/aws/commit/84c28ba16f238a8c48c54a9e858ba4bb2ec53742">commit 84c28ba</a>.</p>
<p>Just a quick overview of other parts:</p>
<ul>
<li>
<p>ELB: This can distribute load among multiple web servers. Even with just one server (to start) it is a convenient way to handle SSL.</p></li>
<li>
<p>SES: I’m only sending “transactional” emails (for account creation, password reset, and search alerts) so it’s been easy so far to maintain a good reputation.</p></li>
<li>
<p>Docker and ECS. Very helpful: <a href="https://www.ybrikman.com/writing/2015/11/11/running-docker-aws-ground-up/"><em>Running Docker on AWS from the ground up</em></a>.</p></li>
<li>
<p>CloudWatch Logs:</p>
<ul>
<li>
<p>It is pretty easy to make a Racket <a href="https://docs.racket-lang.org/reference/logging.html#%28def._%28%28quote._~23~25kernel%29._make-log-receiver%29%29">log receiver</a> that accumulates things into batches and does <a href="https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_PutLogEvents.html"><code>PutLogEvents</code></a>.</p></li>
<li>
<p>It is handy to use JSON format for request/response logs. Not only does this display nicely, there’s a decent query feature, e.g. <code>{$.response.duration > 100}</code> or <code>{$.request.headers.Host =
"deals.extramaze.com"}</code>.</p></li></ul></li></ul>
<h1 id="conclusion">Conclusion</h1>
<p>I hope this helps give a taste of what it’s like to start a small SaaS business c. 2018 using Racket, PostgreSQL, and AWS — but without using advertising or JavaScript.</p>
<p>I realize this post has a somewhat uneven level of detail, so maybe I will loop back later and drill down on some parts.</p>
<div class="footnotes">
<ol>
<li id="extramaze-llc-using-racket-postgresql-aws-but-no-ads-or-js-footnote-1-definition" class="footnote-definition">
<p>I’m simplifying for narrative flow. Day zero, the <em>product</em> name was Cakewalk. The company name was Twelve Tone Systems. Later we adopted Cakewalk as the company name, too. The point is, it was much easier to pick domain names in ye olden times. <a href="#extramaze-llc-using-racket-postgresql-aws-but-no-ads-or-js-footnote-1-return">↩</a></p></li>
<li id="extramaze-llc-using-racket-postgresql-aws-but-no-ads-or-js-footnote-2-definition" class="footnote-definition">
<p>UPDATE: When I wrote this, I overlooked that I was using Google-served fonts. Since then <a href="/2018/05/extramaze-llc-using-system-fonts-not-google-fonts.html">I stopped</a>. <a href="#extramaze-llc-using-racket-postgresql-aws-but-no-ads-or-js-footnote-2-return">↩</a></p></li></ol></div>
<footer></footer></article></div>Racket Makefilesurn:www-greghendershott-com:-2017-04-racket-makefiles.html2017-04-18T16:00:00Z2017-04-18T16:00:00ZGreg Hendershott
<div>
<article>
<header>
<h1>Racket Makefiles</h1>
<p class="date-and-tags">
<time datetime="2017-04-18" pubdate="true">2017-04-18</time> :: <span class="tags"><a href="/tags/Racket.html">Racket</a></span></p></header>
<p>A few years ago <a href="/2014/06/does-your-racket-project-need-a-makefile.html">I wrote about makefiles for Racket</a>. Some things have changed.</p>
<ol>
<li>
<p>The old makefile built and pushed documentation to a GitHub Pages branch of the repo. That’s no longer necessary: The Racket package catalog builds and hosts documentation.</p></li>
<li>
<p>The Racket package catalog puts a yellow badge of shame on packages with missing dependencies (<code>deps</code> and <code>build-deps</code> in the package’s <code>info.rkt</code>). I want the makefile to check this.</p></li>
<li>
<p>In <code>.travis.yml</code> files for Travis CI, I think the <code>script</code> section ought to simply invoke targets in the makefile — delegating details to the latter.</p></li>
<li>
<p>Likewise some details needn’t even be in the makefile — they can move to the collection’s <code>info.rkt</code>. Example: The list of directories to <code>clean</code>.</p></li>
<li>
<p>The old makefile had separate <code>PACKAGENAME</code> and <code>COLLECTS</code> variables; for single-collection packages they were the same value. I wanted to simplify this to just the package name and use the appropriate package variants of <code>raco</code> commands.</p></li></ol>
<p>In that spirit, here’s an updated Makefile, which I recently started using in the <a href="https://github.com/greghendershott/rackjure">rackjure</a>, <a href="https://github.com/greghendershott/markdown">markdown</a>, and <a href="https://github.com/greghendershott/frog">frog</a> projects.</p><!-- more-->
<div class="brush: make">
<table class="sourcetable">
<tbody>
<tr>
<td class="linenos">
<div class="linenodiv">
<pre> 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39</pre></div></td>
<td class="code">
<div class="source">
<pre><span></span><span class="nv">PACKAGE-NAME</span><span class="o">=</span>rackjure
<span class="c"># Racket 6.1 adds pkg dep checking.</span>
<span class="cp">ifeq ($(findstring "$(RACKET_VERSION)", "6.0", "6.0.1"),)</span>
DEPS-FLAGS<span class="o">=</span>--check-pkg-deps --unused-pkg-deps
<span class="cp">else</span>
DEPS-FLAGS<span class="o">=</span>
<span class="cp">endif</span>
<span class="nf">all</span><span class="o">:</span> <span class="n">setup</span>
<span class="c"># Primarily for use by CI.</span>
<span class="c"># Installs dependencies as well as linking this as a package.</span>
<span class="nf">install</span><span class="o">:</span>
raco pkg install --deps search-auto
<span class="nf">remove</span><span class="o">:</span>
raco pkg remove <span class="k">$(</span>PACKAGE-NAME<span class="k">)</span>
<span class="c"># Primarily for day-to-day dev.</span>
<span class="c"># Note: Also builds docs (if any) and checks deps.</span>
<span class="nf">setup</span><span class="o">:</span>
raco setup --tidy --avoid-main <span class="k">$(</span>DEPS-FLAGS<span class="k">)</span> --pkgs <span class="k">$(</span>PACKAGE-NAME<span class="k">)</span>
<span class="c"># Note: Each collection's info.rkt can say what to clean, for example</span>
<span class="c"># (define clean '("compiled" "doc" "doc/<collect>")) to clean</span>
<span class="c"># generated docs, too.</span>
<span class="nf">clean</span><span class="o">:</span>
raco setup --fast-clean --pkgs <span class="k">$(</span>PACKAGE-NAME<span class="k">)</span>
<span class="c"># Primarily for use by CI, after make install -- since that already</span>
<span class="c"># does the equivalent of make setup, this tries to do as little as</span>
<span class="c"># possible except checking deps.</span>
<span class="nf">check-deps</span><span class="o">:</span>
raco setup --no-docs <span class="k">$(</span>DEPS-FLAGS<span class="k">)</span> <span class="k">$(</span>PACKAGE-NAME<span class="k">)</span>
<span class="c"># Suitable for both day-to-day dev and CI</span>
<span class="nf">test</span><span class="o">:</span>
raco <span class="nb">test</span> -x -p <span class="k">$(</span>PACKAGE-NAME<span class="k">)</span>
</pre></div>
</td></tr></tbody></table>
</div>
<p>The two main scenarios here:</p>
<ul>
<li>
<p>Day-to-day development: <code>make setup</code> and <code>make test</code>.</p></li>
<li>
<p>CI: <code>make install</code>, <code>make check-deps</code>, and <code>make test</code>.</p></li></ul>
<p>I think you could probably use this as a template for any single-collection package project. Just change <code>PACKAGE-NAME</code>. Possibly append a target or two for something unique to your project.</p>
<hr />
<p>This Makefile is designed to work with Racket 6.0 or newer — because I have some existing packages that support Rackets that old. If you only care about Racket 6.1 or newer, then all this:</p>
<div class="brush: make">
<table class="sourcetable">
<tbody>
<tr>
<td class="linenos">
<div class="linenodiv">
<pre>1
2
3
4
5
6</pre></div></td>
<td class="code">
<div class="source">
<pre><span></span><span class="c"># Racket 6.1 adds pkg dep checking.</span>
<span class="cp">ifeq ($(findstring "$(RACKET_VERSION)", "6.0", "6.0.1"),)</span>
DEPS-FLAGS<span class="o">=</span>--check-pkg-deps --unused-pkg-deps
<span class="cp">else</span>
DEPS-FLAGS<span class="o">=</span>
<span class="cp">endif</span>
</pre></div>
</td></tr></tbody></table>
</div>
<p>can become just this:</p>
<div class="brush: make">
<table class="sourcetable">
<tbody>
<tr>
<td class="linenos">
<div class="linenodiv">
<pre>1</pre></div></td>
<td class="code">
<div class="source">
<pre><span></span><span class="nv">DEPS-FLAGS</span><span class="o">=</span>--check-pkg-deps --unused-pkg-deps
</pre></div>
</td></tr></tbody></table>
</div>
<hr />
<p>By the way, I wouldn’t call myself a very experienced user of <code>make</code>.</p>
<p>For example the way I check for Racket < 6.1 seems smelly — but it was the least-worst way I could figure out.</p>
<p>So please feel free to share any suggestions or corrections in the comments.</p>
<footer></footer></article></div>