published
updated
It's been a while since I worked on my December Adventure, but we're back baby! Much to my surprise, I'm both able and willing to continue ▓▘▙▓▝ing on the parser. I wrote the inline footnote reference parser, and fixed a couple bugs along the way. I'm somewhat surprised I was able to pick up the mental model for this parser as quickly as I did after 10 days of not touching it. That's a great sign! I left enough comments in the file itself to help me understand what I needed to, and there were some helper functions I'd previously used to debug things which came in handy. I need to implement some sort of "debug" view, or rather find a pattern for debug view because currently it always spits out a heap of debug information. Did I mention I added more debug information?
I'm really not looking forward to writing tests for this. It still feels "fragile", any erroneous input could lead to total parse failure which is annoying, but needs to be accounted for. This is going to be a chore, but it's necessary for me to do, since this parser will eventually become a foundational part of the rest of my website, and perhaps even Kitaab generally. I like the idea of writing a Glow but that's a big maybe at this stage.
So now that now leaves the render stage of this parser. I still need to wire up footnotes to their associated links, but I have a basic idea for how to do that now, since I've parsed inline footnote refs. The ref itself contains a number or the # character. We keep an internal counter that increments every time we come across a #, and link to that footnote number, and if it's a number we simple link to that specific number. We use the number as an id when generating the footnotes section and la-di-da, working footnotes links.
God it feels nice to be programming again :3
Today I realized that I haven't thought about footnotes enough. I hadn't even parsed them with the inline parser 😕 Whoops... Even though I'm parsing footnote blocks, I'm not yet certain of how to link inline footnote references to a specific footnote (which is a line) in the footnote block. I'm not sure how to be able to specify order, though having it be auto-ordered sounds do-able. Which is also making me realize that I don't have numbered lists as part of my parser. I personally don't really use numbered lists, but it could be useful here? In terms of code writing, I did sort out how to deal with metadata in the final render, and I decided that metadata doesn't get rendered. We could do it at the in-line lexing stage? It could also be the final stage, it doesn't matter too much, and kind of flows well with what I've got now anyway. I've put in the scaffolding for metadata parsing in the renderer, though it doesn't handle any logic for each section 🤔 I find it's important for me to think a little more about the design of my parser / final renderer before I commit to a design, so progress has been a little slower lately.
It is coming along nicely though, and I've started thinking about what I want to write next. I'm not yet convinced that I'll write the SSG to replace basant that incorporates this just yet. I am having a lot of fun writing Fennel, but I might switch gears to Go. I mostly want to use BubbleTea to build a small pomodoro / task TUI program. I originally thought of writing this is Janet but then I'd have to learn about ANSI Escape codes and all that jazz, and really I just want a simple pomodoro TUI app that integrates with Taskwarrior and the rest of my system (like a lil text in my polybar reminding me of the current task), and I've been meaning to learn Go
Look at me go though, always ready to think about the next project before the current one is complete. The project isn't complete until the tests are written!!
Day 6 and we're on to rendering ✨
So far I wrote the structure to render most the specific blocks. I still need to write the contents renderer that will handle in-line rendering.
I'm also unsure on how to handle metadata (and tags, which are a form of metadata) or code segments. At least for code segments I can look into pygments or some other JS solution (with some sort of progressive enhancement?). For metadata I'm thinking that it shouldn't actually be rendered, and passed back up for whatever other consumer to deal with (Likely an SSG on top) I did write about how to handle metadata and sections, so this seems like a reasonable place to do that?
Day 4 and I didn't spend a heap of time working on my lil parser today, but I had enough scaffolding setup that I could go in and fix my content-lexer in about 10 minutes :)
The only issue I can spot (I still need to write test cases for both lexers currently...) is that empty paragraphs aren't handled gracefully. Sometimes the contents are empty maps, sometimes they get parsed as strings containing only "\n", which will make rendering more annoying.
Day 3, and I've got the inline parser working, but the decoration step is not quite correct. But that'll be tomorrow's problem. I don't think the renderer will be too difficult, or at least I hope not. I'm kinda happy with the way it all came together. Once the parser is written out, I'll tighten up the spec to be exact. I'm not sure that the way I've written my parser is particularly efficient or a good way to do this. I'd like to experiment with writing it in a combinatory style, but I don't know that I'll get to that. Look at me, already discussing rewrites before I've even completed the first one.
I've also realized that given the current implementation of decoration, I can't stack decorations. Which is annoying, but not really something I do anyway? Ah well.
I discovered this while reading the spec for mycomarkup which supports stacking decorations and also includes transclusions! I think transclusions are really neat, and even though I don't currently use them, I can already think of places where this would be a good idea. I think I could use these in my index generation, where I want to be able to describe each tag, but perhaps it could also come in handy while Link Logging.
Started writing the inline lexer sort of, but not sure I'm doing it correctly. Since we now know the "type" of each block, we can extract out semantically meaningful data. The one thing I'm still unsure about is how to apply decoration checks to all contents. Regardless of it's a header, a paragraph, or a list, I should still be able to apply any of the decorations I can define. Image alt descriptions should also be decorated, and so too should footnote content.
I'm also thinking about whether I want to be able to link to specific sections. The simplest way to implement this wouold be to have a "reverse" counter: the last parsed section is 0 and we count upwards to the most recent. Since by convention we insert data at the "start" of the file. I could have a special character (I was thinking #) to refer to a section, but that means the filename cannot have that character. Personally, that seems okay?
I'm also currently unsure how I'm to link footnotes against their reference.
https://git.sealight.xyz/aynish/incd-parser
== I'm finally Writing a Parser ==
I first wrote about writing a Parser nearly 6 months to the day ago. But hey, now I'm doing it 😎
I decided to do this as part of eli_oat's December Aadventure. I'd never really been one to do Advent of Code, but I do enjoy programming my own tools a bunch. I'd already been working on some other personal tool in Fennel as part of this website build pipeline, and now that those are complete, it made sense to move on to writing a parser.
I managed to write the block lexer elements, but it still needs tests for me to feel confident in it.