From: Kyle Bowman Date: Sun, 15 Dec 2024 01:37:23 +0000 (-0500) Subject: refactor: clean up cruft and reorganize X-Git-Url: https://git.rocketbowman.com/?a=commitdiff_plain;h=5090b521180419c22b1d997146e2476988a0b609;p=jetsam.git refactor: clean up cruft and reorganize --- diff --git a/.gitignore b/.gitignore index d402616..52964d7 100644 --- a/.gitignore +++ b/.gitignore @@ -1,3 +1,3 @@ .buildlist build/ -src/ +src diff --git a/Makefile b/Makefile index 2cd9a09..b842ed8 100644 --- a/Makefile +++ b/Makefile @@ -7,7 +7,7 @@ BUILD_LIST := $(shell find -L . -path './$(SRC_DIR)/**/*.md' | sed 's:./$(SRC_DI help: @echo "Builds HTML files from MD files. Targets include:" @echo - @echo "all Builds the .buildlist and an index. (Top-level)" + @echo "all Builds the .buildlist and an index.html. (Top-level)" @echo "clean Removes build artifacts." @echo "help Prints this help message." @echo ".buildlist Defines a list of all files to build. Paths may be relative." @@ -15,6 +15,9 @@ help: @echo " may create this file manually or override the BUILD_LIST variable when" @echo " you invoke the make command. The default BUILD_LIST is created from MD files" @echo " found anywhere under the $(SRC_DIR) directory." + @echo "init-demo Creates links from demo to $(SRC_DIR) so you can then build it." + @echo "init-altdemo Creates links from demo to $(SRC_DIR) (alternate method) so you can then build it." + @echo "rm-src Deletes symlinks in $(SRC_DIR), checking for non-link data." .PHONY: all: build/index.html @@ -32,13 +35,7 @@ $(BUILD_DIR)/$(ASSETS_DIR): $(SRC_DIR)/$(ASSETS_DIR) $(BUILD_DIR)/%.html: $(SRC_DIR)/%.md $(BUILD_DIR)/$(ASSETS_DIR) @mkdir -p $(dir $@) - @pandoc \ - --to=html5 \ - --standalone \ - --toc=true \ - --defaults=./defaults.yml \ - --output=$@ $< \ - --lua-filter=scripts/links-to-html.lua + @pandoc --defaults=./defaults/md-to-html.yml --output=$@ $< @echo "Building $@ from $<..." .PHONY: diff --git a/defaults.yml b/defaults.yml deleted file mode 100644 index 86d3076..0000000 --- a/defaults.yml +++ /dev/null @@ -1,6 +0,0 @@ -from: markdown+wikilinks_title_before_pipe -css: - # ${.} is robust on build, but probably not for deployment - - ../assets/css/normalize.css - - ../assets/css/space-sakura.css - diff --git a/defaults/md-to-html.yml b/defaults/md-to-html.yml new file mode 100644 index 0000000..4cc8b71 --- /dev/null +++ b/defaults/md-to-html.yml @@ -0,0 +1,9 @@ +from: markdown+wikilinks_title_before_pipe +to: html5 +filters: + - ${.}/../filters/links-to-html.lua +standalone: True +toc: True +css: + - ../assets/css/normalize.css + - ../assets/css/space-sakura.css diff --git a/demo.html b/demo.html deleted file mode 100644 index 6b39cae..0000000 --- a/demo.html +++ /dev/null @@ -1,209 +0,0 @@ - - - - - sakura demo - - - - - -

-

Space Sakura Demo

-

This is an anchor tag/hyper link: sakura - Github Repository

-

Paragraphs look like this. Font size along with line height and maximum width are optimized for reading.

- -

Italic, bold, and monospace. Itemized lists look like:

-

this one
that one
the other one

- -

Here's a block quote:

-

-
Man surprised me most about humanity. Because he sacrifices his health in order to make money. Then he sacrifices money to recuperate his health. And then he is so anxious about the future that he does not enjoy the present; the result being that he does not live in the present or the future; he lives as if he is never going to die, and then dies having never really lived. -James J Lachard
-

- -

An h2 header

-

Some code blocks

-

define foobar() {
-    print "Welcome to flavor country!";
-}

-

-

import time
-# Quick, count to ten!
-for i in range(10):
-    # (but not *too* quick)
-    time.sleep(0.5)
-    print i

-

- -

An h3 header

-

A nested list:

-

-
First, get these ingredients:
-
- carrots
- celery
- lentils
-
-
Boil some water.
-

-

Dump everything in the pot and follow this algorithm:

-

find wooden spoon
-uncover pot
-stir
-cover pot
-balance wooden spoon precariously on pot handle
-wait 10 minutes
-goto first step (or shut off burner when done)

-

Do not bump wooden spoon or it will fall.

-

- -

Header level 1

-

Header level 2

-

Header level 3

-

Header level 4

-

Header level 5

-

Header level 6

- -

A horizontal line:

-

- -

Here's a link to a website, to a local doc, and to a section heading in the current doc. Here's a footnote ¹.

-

Tables can look like this:

- - - - - - - - - - - - - - - - - - - - - - - - - - - -

- Shoes, their sizes, and what they're made of -
size	material	color
9	leather	brown
10	hemp canvas	natural
11	glass	transparent

-

Multi-line tables:

- - - - - - - - - - - - - - - - - - - - - -

keyword	text
red	Sunsets, apples, and other red or reddish things.
green	Leaves, grass, frogs and other things it's not easy being.

- -

A horizontal rule follows.

-

-

Images are responsive by default:

-

-

example image

-

example image

-

- -

- -

Form Elements

-

-

-

Input fields

-

Text Input

-

Password

-

Web Address

-

Email Address

-

Phone Number

-

Search

-

Number Input

-

Error

-

Valid

-

-

-

-

Select menus

-

Select

-

-

-

-

Checkboxes

-

Choice A
Choice B
Choice C

-

-

-

-

Textareas

-

Textarea -

-

-

-

-

-

Action buttons

-

-

-

-

-

-

-

-

-

Footnote text goes here.â©

-

-

- - diff --git a/experiments.md b/experiments.md deleted file mode 100644 index 11a3c6b..0000000 --- a/experiments.md +++ /dev/null @@ -1,119 +0,0 @@ -# Experiments with Publishing My Markdown Notes - -## Summary - -* Turns out I don't need an app, I need a renderer. -* Pandoc can render Markdown well. And now, it supports wikilinks. I might be -able to have "internal" linking. -* Find | Pandoc can work with a weird lambda-function kind of approach. -* Make files also work pretty well. -* I like sakura.css as a starting point: https://github.com/oxalorg/sakura -for styling. - -## Find | Pandoc - -Single doc: - -``` bash -pandoc --from=markdown+wikilinks_title_before_pipe --to=html5 -o build/note2 src/note2.md -``` - -Note: Pandoc complains if the build directory doesn't already exist. - -I think this should work. It grabs all the files in the src directory. -For each file it finds, run `pandoc ... ` and output to build/note1 or -build/note2. -``` bash -find . -path './src/*' -exec pandoc --from=markdown+wikilinks_title_before_pipe --to=html5 -o build/$(basename -s .md {}) {} \ -``` - -Output -``` -pandoc: build/./src/note2.md: withFile: does not exist (No such file or directory) -pandoc: build/./src/note1.md: withFile: does not exist (No such file or directory) -``` - -To confirm that the basename shenanigans is doing what I want: -``` bash -find . -path './src/*' -exec basename -s .md {} \; -# Output: note2\nnote1 -``` - -Furthermore, without the output specified, it concatenates the input files and writes to stdout: -``` -find . -path './src/*' -exec pandoc --from=markdown+wikilinks_title_before_pipe --to=html5 {} \; -``` - -This seems to identify the culprit! Something about passing -the found files into a subshell. -``` bash -find . -path './src/*' -exec echo build/$(basename -s .md {}) \; -# Output build/./src/note2.md... -``` - -This does it! -``` bash -find . -path './src/*' -exec sh -c 'pandoc --from=markdown+wikilinks_title_before_pipe --to=html5 -o build/$(basename -s .md $1) $1' _ {} \; -``` - -This is a pattern that I didn't expect for shells, but it's used elsewhere. -It's like defining a lambda function and applying it in one step. Only we are -using a subshell to define the lambda function. Here's the source from -[Stack Overflow](https://stackoverflow.com/questions/13833824/how-to-run-a-subcommand-inside-find-exec). - -1. In a subshell, *define a function* that you want to run. -2. When you close the subshell, the rest of the exec function applies the function. - -There is no passing the `{}` into the subshell. The `{}` exists in the original -`-exec` shell. Instead, you're doing some kooky function definition. - -Note: We are using `$1` to align with the convention that `$0` is the name and -`$1` is the first argument. - -From the man page of bash (emphasis mine): - ->-c If the -c option is present, then commands are read from the first -non-option argument command_string. If there are arguments after the command_string, *the first argument is assigned to $0 and any remaining -arguments are assigned to the positional parameters*. The assignment to $0 sets -the name of the shell, which is used in warning and error messages. - -# Regarding Map in Bash - -Map would be nice because then you don't have to loop inside definitions -But looping doesn't play nice with spaces. -I was pleasantly suprised it worked. Maybe there's a way around the -space issue, but today isn't the day for that. - -``` bash -map() { - for item in $2; do - # shellcheck disable=SC2005 - echo "$("$1" "$item")" - done -} -``` - -## BUG: Map breaks on spaces -Context `map "make_li" "$1"`. - -When `$1` contains a space, it breaks there. *e.g.* when it contains -`\n

` -and so on. To be clear, this is not the fault of `make_li`. -This is due to `$1` begin a space-delimited list and `map` breaking -on spaces. Also, to be fair, that's just the way shells work by default. -There's probably a clever way to manipulate lists as strings, but I don't know -what the idiomatic shell way of doing it is. - -## Cut Fiasco -`cut ... $1` reads the file itself. -`echo $1 | cut ...` reads the string in `$1`. - -# Conclusion -After tidying it all up, this command builds my entire stack of HTML and the links work! - -``` bash -find ~/notes/rocketbowman/tiddlers/ -path '*.md' -exec sh -c 'pandoc --from=markdown+wikilinks_title_before_pipe --to=html5 -o ~/tmp/build/$(basename -s .md $1) $1' _ {} \; -``` - -## A better conclusion -Turns out that a Makefile is the way to go. Create a rule for a 1-1 conversion. Then add a `make all` command. \ No newline at end of file diff --git a/filters/links-to-html.lua b/filters/links-to-html.lua new file mode 100644 index 0000000..2de3fd0 --- /dev/null +++ b/filters/links-to-html.lua @@ -0,0 +1,6 @@ +-- links-to-html.lua +-- From https://stackoverflow.com/a/49396058 +function Link(el) + el.target = string.gsub(el.target, "%.md", ".html") + return el +end \ No newline at end of file diff --git a/link-dependencies.py b/link-dependencies.py deleted file mode 100644 index 593bd8a..0000000 --- a/link-dependencies.py +++ /dev/null @@ -1,86 +0,0 @@ -import subprocess -import re -from pathlib import Path - -# Motivating Question: Can I develop a dependency list for a file based on -# the wikilinks within? - - -def extract_substrings(input_string): - # Get substrings between [[ and ]] - pattern = r'\[\[(.*?)\]\]' - matches = re.findall(pattern, input_string) - return matches - -def process_strings(string_list): - # If we are taking everything up to the pipe, we don't need if/else. - return [s.split('|')[0] for s in string_list] - -def test_extract_substrings(): - assert extract_substrings("Here are some substrings: [[first]], [[second]]") == ["first", "second"] - -def test_process_strings(): - assert process_strings(["wikilink|title","WikiLinkOnly"]) == ["wikilink","WikiLinkOnly"] - -def test_integration(): - extracted = extract_substrings("Titled Wikilink: [[wikilink|title]]; untitled Wikilink: [[WikilinkOnly]]") - processed = process_strings(extracted) - assert processed == ["wikilink", "WikiLinkOnly"] - -def run_pandoc(input_file, output_file, format='html'): - """ - Runs the pandoc command to convert a file to a different format. - - Parameters: - - input_file (str): Path to the input file. - - output_file (str): Path to the output file. - - format (str): Desired output format. Default is 'pdf'. - """ - - # Construct the command - command = [ - 'pandoc', - input_file, - '-o', - output_file, - '-t', - format - ] - - try: - result = subprocess.run(command, check=True, text=True, capture_output=True) - - # Print stdout and stderr - print("stdout:", result.stdout) - print("stderr:", result.stderr) - - except subprocess.CalledProcessError as e: - print("Error occurred while running pandoc:") - print(e.stderr) - except FileNotFoundError: - print("The 'pandoc' command was not found. Please make sure it is installed and in your PATH.") - except Exception as e: - print(f"An unexpected error occurred: {e}") - -# Example usage - - -if __name__=="__main__": - #test_extract_substrings() - #test_process_strings() - #print("all tests passed") - - base_dir = Path("/home/kyle/notes/rocketbowman/tiddlers") - file = base_dir / "ElectricityAndMagnetism.md" - with open(file, "r") as f: - content=f.read() - raw = extract_substrings(content) - processed = process_strings(raw) - print(f"Raw strings: {raw}") - print(f"Processed strings: {processed}") - - # Run pandoc - - input_path = base_dir / (processed[1] + ".md") - output_path = f"/home/kyle/tmp/{processed[1]}" - run_pandoc(input_path, output_path) \ No newline at end of file diff --git a/scripts/combine.sh b/scripts/combine.sh deleted file mode 100755 index 8861250..0000000 --- a/scripts/combine.sh +++ /dev/null @@ -1,16 +0,0 @@ -#!/bin/bash -basenames="$(find ./tiddlers -name '*.md' -exec basename -s .md {} \;)" -outdir="./test" - -if [ ! -d "$outdir" ]; then - mkdir "$outdir" -fi; - -for file in $basenames; do - { - echo "---" - cat "./tiddlers/$file.md.meta" - echo; echo "---"; echo; - cat "./tiddlers/$file.md" - } > "$outdir/$file.md" -done \ No newline at end of file diff --git a/scripts/links-to-html.lua b/scripts/links-to-html.lua deleted file mode 100644 index 2de3fd0..0000000 --- a/scripts/links-to-html.lua +++ /dev/null @@ -1,6 +0,0 @@ --- links-to-html.lua --- From https://stackoverflow.com/a/49396058 -function Link(el) - el.target = string.gsub(el.target, "%.md", ".html") - return el -end \ No newline at end of file diff --git a/templates/deprecated.base.html b/templates/deprecated.base.html deleted file mode 100644 index a5b07eb..0000000 --- a/templates/deprecated.base.html +++ /dev/null @@ -1,44 +0,0 @@ -$-- This is a comment; Use $$ for literal $ -$-- Details for writing this template can be found: https://pandoc.org/MANUAL.html#templates -$-- ${foo} or $foo$ is an interpolated variable -$-- variables come from YAML front matter (or command line) -$-- ${if(variable)} conditional text ${endif} -$-- ${for(foo)} do thing $foo$ ${endfor} -$-- ${template-name.html} includes another template -$-- You can use pipes to transform variables if needed - - - - - $title$ - - -$for(author-meta)$ - -$endfor$ -$if(date-meta)$ - -$endif$ -$if(description)$ - -$endif$ -$if(tags)$ - -$endif$ -$-- - - - -

-

$title$

-

-$body$ - - \ No newline at end of file diff --git a/todo.md b/todo.md deleted file mode 100644 index 7630645..0000000 --- a/todo.md +++ /dev/null @@ -1,84 +0,0 @@ -# Design philosophy -* Few dependencies - * use pandoc features itself when possible - * use pandoc filters for structured processing - * use pandoc-lua for general-purpose scripting -* Separate framework from content. - * You can link content into the main content directory - -# To Do - -## Figure out Images - -* Images are part of content and should be linked in. -* Images might be their own thing or part of a subdirectory. (e.g. public/private) -* Figuring out how, in the Makefile, to copy images/assets might be tricky. - -## Improve Index - -* index.sh is hardcoded and janky - it should probaby leverage a pandoc-lua - and/or pandoc templates. -* (Stretch) Should acknowledgements be a template? - - -## Navigation Improvements: -* Enhancement: Create an index/nav item per directory (and eventually keywords?) - * (Defer) About - add acknowledgements - * (Defer) Summaries - index - * (Defer) Figure out other categories. - -## Make Deploy -* Figure out a `make deploy` with a `.env` file. - * Serve on push. (If using git hook in rb, use .env file) - -## Clean up / Refactor -* Try to decouple hardcoded things - * index and template use different hardcoded paths to CSS. - * (Stretchier) Should I mention Pandoc (def), GNU Make (prob), VSCodium (prob not), Foam (prob not), Debian(very prob not)? -* Can I centralize config? A `make config` would be nice if it includes - static content (like `author="Kyle Bowman"`) and dynamic content like - `files=$(ls build)`. -* Index.html will eventually need to be renamed b/c it's an "index", not the landing page. -* (Stretch) Reformulate my templates based on jgm's templates so I can - merge updates easily. -* Extract lessons from `experiments.md`. - * I'm torn about where to put these. In the project or in my notes? -* (Stretch) Formalize demo and add tests -* (Stretch) Add a hotkey to strike through a line in a to-do list. -* (Stretch) Start building a CLI library that helps me manage md files. - * Hard wrap the MD files themselves. - * When a bullet point wraps, the first letter of the wrapped line ought to - align with the first letter of the leading line instead of aligning with - the bullet point. (Like this really long bullet point.) - * Spell check - -# Feature: Interactive Terminal/Markdown Experience - -I want my editor to be able to send and receive info from my terminal in -order to create an interactive writing/experimenting experience. Here are some -ideas: - -* For many things, I want to target a line, for others an inline element, -and still others, a block element. -* From an editor, I want to be able to do the following to each target - * Copy a target to the clipboard - * Send a target to the terminal (and run it?) -* From a terminal, I want to be able to: - * Copy a command to the clipboard - * Send a command to the active tab. - * Copy the output to a clipboard - * Send stdout to the active tab. - -# Feature: Tangle and Weave -* When rendering my notes, I want to be able to tangle and weave the notes in -the sense of Knuth's Web literate programming framework. - * Weave creates doc from source - * Tangle creates a program from source - * Code blocks should support filename - * Bash should support setting PWD before running. (And probably env vars - in general.) - * I have no idea how this would work. - * Maybe, you ought to just supply a .env file whenever you run the - command to extract the code. - * Both should be able to be supplied as a variable so it's executable from - another machine. \ No newline at end of file