/*@jsxRuntime automatic @jsxImportSource react*/ const {Fragment: _Fragment, jsx: _jsx, jsxs: _jsxs} = arguments[0]; const {useMDXComponents: _provideComponents} = arguments[0]; function _createMdxContent(props) { const _components = Object.assign({ p: "p", h2: "h2", pre: "pre", code: "code", h3: "h3", a: "a" }, _provideComponents(), props.components); return _jsxs(_Fragment, { children: [_jsx(_components.p, { children: "This was a quick hack over the weekend, none of this code should be taken as production grade or as idiomatic code in Rust or Go. I've been learning Rust for a bit and it's been an interesting and frustrating experience. I was recently encouraged to take a look a Go. I hadn't tried Go in years, the aesthetics of the language had put me off. I thought it looked like what C should have been in 2000 but not a \"modern language\". So, I decided to give it a fair chance and try Go, Rust, and TypeScript. Here are the results." }), "\n", _jsx(_components.h2, { children: "The challenge" }), "\n", _jsx(_components.p, { children: "I've been building a small app to manage my subscriptions to email newsletters (more on that soon). As part of that, I downloaded a large number of email newsletters from my Gmail and want to extract links. This will be useful for spotting trends, search, and a few other applications. So the initial challenge is to load a JSON file of exported messages with this format:" }), "\n", _jsx(_components.pre, { children: _jsx(_components.code, { className: "language-json", children: "[\n {\n \"id\": \"fc62fdcd-e8ce-4f54-a7c3-d28a56d3d663\",\n \"fromEmail\": \"news@thehustle.co\",\n \"fromName\": \"The Hustle\",\n \"htmlBody\": \" {\n const target = link.getAttribute(\"href\");\n if (target && target.startsWith(\"http\")) {\n links.push(target);\n }\n });\n\n return links;\n}\n\n(async () => {\n const startTime = new Date().getTime();\n\n const messages: Message[] = JSON.parse(\n fs.readFileSync(\"../messages.json\", \"utf-8\")\n );\n const allLinks: string[] = [];\n\n messages.forEach((message) => {\n if (message.htmlBody) {\n const links = extractLinks(message.htmlBody);\n if (links.length > 0) {\n allLinks.push(...links);\n }\n }\n });\n\n fs.writeFileSync(\"./links.txt\", allLinks.join(\"\\n\"));\n\n const endTime = new Date().getTime();\n console.log(`time: ${endTime - startTime}ms`);\n})();\n" }) }), "\n", _jsx(_components.h3, { children: "Go" }), "\n", _jsx(_components.p, { children: "Well, that was easy." }), "\n", _jsx(_components.p, { children: "With no prior knowledge of Go beyond writing \"Hello, World!\" a few years ago, I was able to read the docs, Google some things, and get this working in about 10-15min. Everything worked as expected on the first try and was clearly much faster than the TypeScript solution." }), "\n", _jsx(_components.pre, { children: _jsx(_components.code, { className: "language-go", children: "package main\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"io/ioutil\"\n\t\"log\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/PuerkitoBio/goquery\"\n)\n\ntype Message struct {\n\tId string `json:\"id\"`\n\tFromEmail string `json:\"fromEmail\"`\n\tFromName string `json:\"fromName\"`\n\tHtmlBody string `json:\"HtmlBody\"`\n}\n\nfunc ExtractLinks(html string) []string {\n\tdoc, err := goquery.NewDocumentFromReader(strings.NewReader(html))\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\tvar links []string\n\n\tdoc.Find(\"a\").Each(func(i int, s *goquery.Selection) {\n\t\ttarget, exists := s.Attr(\"href\")\n\t\tif exists {\n\t\t\tlinks = append(links, target)\n\t\t}\n\t})\n\n\treturn links\n}\n\nfunc main() {\n\tstart := time.Now()\n\n\tcontent, err := ioutil.ReadFile(\"../messages.json\")\n\tif err != nil {\n\t\tfmt.Println(err)\n\t}\n\n\tvar payload []Message\n\terr = json.Unmarshal(content, &payload)\n\tif err != nil {\n\t\tlog.Fatal(\"Error during Unmarshal(): \", err)\n\t}\n\n\tvar alllinks []string\n\n\tfor _, value := range payload {\n\t\talllinks = append(alllinks, ExtractLinks(value.HtmlBody)...)\n\t}\n\n\tioutil.WriteFile(\"links.txt\", []byte(strings.Join(alllinks, \"\\n\")), 0644)\n\n\tduration := time.Since(start)\n\tfmt.Printf(\"time: %dms\\n\", duration.Milliseconds())\n}\n" }) }), "\n", _jsx(_components.h3, { children: "Rust" }), "\n", _jsx(_components.p, { children: "Well, that was hard." }), "\n", _jsx(_components.p, { children: "Rust continues to make me feel like I'm just not smart enough to program in it. Getting a working solution was a battle at every step along the way. It took lots of reading Stack Overflow and trying to see what the compiler let me get away with to get it working. This easily took 45min to an hour which was the longest development time of any solution. Looking back on it, I still barely understand all the pieces." }), "\n", _jsx(_components.pre, { children: _jsx(_components.code, { className: "language-rust", children: "use serde::Deserialize;\nuse std::error::Error;\nuse std::fs::File;\nuse std::io::{BufReader, Write};\nuse std::path::Path;\nuse std::time::Instant;\n\n#[derive(Debug, Deserialize)]\n#[serde(rename_all = \"camelCase\")]\n#[allow(dead_code)]\nstruct Message {\n id: String,\n from_email: String,\n from_name: String,\n html_body: String,\n}\n\nfn load_messages>(path: P) -> Result, Box> {\n let file = File::open(path)?;\n let reader = BufReader::new(file);\n let messages: Vec = serde_json::from_reader(reader)?;\n\n Ok(messages)\n}\n\nfn extract_links(html: &str) -> Result, Box> {\n let dom = tl::parse(html, tl::ParserOptions::default()).unwrap();\n let parser = dom.parser();\n let links = dom\n .query_selector(\"a[href]\")\n .unwrap()\n .map(|e| e.get(parser).and_then(tl::Node::as_tag).unwrap())\n .map(|e| {\n e.attributes()\n .get(\"href\")\n .flatten()\n .and_then(tl::Bytes::try_as_utf8_str)\n .unwrap()\n .to_string()\n })\n .collect();\n\n Ok(links)\n}\n\nfn main() -> std::io::Result<()> {\n let now = Instant::now();\n\n let messages = load_messages(\"../messages.json\").unwrap();\n let mut all_links: Vec = vec![];\n\n for message in messages {\n let mut links = extract_links(&message.html_body).unwrap();\n all_links.append(&mut links);\n }\n\n let mut file = File::create(\"links.txt\")?;\n let contents = all_links.join(\"\\n\");\n file.write_all(contents.as_bytes())?;\n\n let elapsed = now.elapsed();\n println!(\"time: {:?}ms\", elapsed.as_millis());\n\n Ok(())\n}\n" }) }), "\n", _jsx(_components.h2, { children: "Benchmarks" }), "\n", _jsx(_components.p, { children: "These are not scientific at all. Perhaps the only lesson here is what out of the box performance looks like for someone hacking at a language the first time with no prior knowledge of any of the performance implications of their actions." }), "\n", _jsx(_components.p, { children: "| Language | Avg. Time |\n| ---------- | --------- |\n| TypeScript | 2688ms |\n| Go | 796ms |\n| Rust | 494ms |" }), "\n", _jsx(_components.h2, { children: "Conclusion" }), "\n", _jsx(_components.p, { children: "Again, I feel a disclaimer is necessary here. This is not a fair benchmark because the time is dominated by HTML parsing and I have no idea what the relative optimization levels of the parsing solutions in each language." }), "\n", _jsx(_components.p, { children: "Go was the surprise here. I expected Rust to be the fastest and Go to be faster than TS but I didn't expect how fast and easy it would be to write code in Go. Based on my experience here, it seems pretty easy to rewrite some performance critical code in Go vs. TS and achieve a pretty big boost. If you're deploying to AWS Lambda or similar solutions that charge for CPU + memory you'd see some pretty big cost savings." })] }); } function MDXContent(props = {}) { const {wrapper: MDXLayout} = Object.assign({}, _provideComponents(), props.components); return MDXLayout ? _jsx(MDXLayout, Object.assign({}, props, { children: _jsx(_createMdxContent, props) })) : _createMdxContent(props); } return { default: MDXContent };

Typescript production deploys with docker

Parsing HTML email

Mermaid MDX component