Jon Gjengset
Jon Gjengset
  • Видео 107
  • Просмотров 4 405 231
Implementing a Lox interpreter in Rust
For some time I've been looking for an opportunity to do a stream on writing a parser + interpreter for... something. Anything really. I've wanted to implement a parser following matklad's excellent article on Pratt parsing ( matklad.github.io/2020/04/13/simple-but-powerful-pratt-parsing.html ), though honestly anything in the area of parsing would do. Well, CodeCrafters have recently released a new challenge in beta that follows Robert Nystrom's Crafting Interpreters book ( craftinginterpreters.com/ ), so I jumped at the opportunity. Join me in working through the challenges, or try it yourself first and compare notes after the fact!
You can access the challenge over at app.codecrafters.i...
Просмотров: 30 702

Видео

Open Source Maintenance, 2024-07-14
Просмотров 21 тыс.Месяц назад
Due to some recent travel (both for fun and for work), I've yet again fallen behind on my GitHub notifications across the various open-source projects I maintain. In this video, like the others like it in the past, we made our way through as many of those as we could over the course of a few hours! And as with the past ones of these, my hope is that by showing the maintainer side of open-source...
Q&A May 2024
Просмотров 20 тыс.4 месяца назад
Audio podcast version: pod.jonhoo.eu/episode/qna/2024-05-04T10-05-43Z/ Links from the stream: - Discord: discord.jonhoo.eu - LogLog Games article: loglog.games/blog/leaving-rust-gamedev/ - Rust API Guidelines: rust-lang.github.io/api-guidelines/ - Rust Design Patterns Book: rust-unofficial.github.io/patterns/ - Data Latam podcast: www.datalatam.com/ - Engineering blogs: - words.filippo.io/dispa...
May 2024 Q&A
Просмотров 4,9 тыс.Месяц назад
Also available as a video on RUclips (ruclips.net/video/diCEj3F3itc/видео.html) . Questions and answers session from May, 2024. Link list • Discord: discord.jonhoo.eu • LogLog Games article: loglog.games/blog/leaving-rust-gamedev/ • Rust API Guidelines: rust-lang.github.io/api-guidelines/ • Rust Design Patterns Book: rust-unofficial.github.io/patterns/ • Data Latam podcast: www.datalatam.com/ •...
Decrusting the tokio crate
Просмотров 90 тыс.5 месяцев назад
In this stream, we peeled back the crust on the tokio crate - github.com/tokio-rs/tokio/ - and explored its interface, structure, and mechanisms. We talked about blocking, cancellation, spawning, and mechanisms for synchronization. We also dug into some of what goes on under the hood where that ends up being relevant to you as an application author! For more details about tokio, see docs.rs/tok...
Implementing (parts of) git from scratch in Rust
Просмотров 81 тыс.5 месяцев назад
In this stream, we implement core pieces of git from scratch by following the CodeCrafters git "course" @ app.codecrafters.io/join?via=jonhoo, just like we did for BitTorrent in ruclips.net/video/jf_ddGnum_4/видео.html. If you sign up with the link above, you get free access to the challenge (and all their challenges) for 7 days. Alternatively, you can access the content for free (albeit withou...
Decrusting the tracing crate
Просмотров 50 тыс.6 месяцев назад
In this stream, we peel back the crust on the tracing crate - github.com/tokio-rs/tracing/ - and explore its interface, structure, and mechanisms. We talk about spans, events, their attributes and fields, and how to think about them in async code. We also dig into into what subscribers are, how they pick up events, and how you can construct your own subscribers through the layer abstraction. Fo...
Hardware and software [2024 edition]
Просмотров 27 тыс.7 месяцев назад
Today's stream (finally) going over my hardware and software setup is now up! And it's not even that long :p GitHub Sponsors: github.com/sponsors/jonhoo/ Discord server: discord.jonhoo.eu/ Software configs: github.com/jonhoo/configs Hardware: Monitor: www.gigabyte.com/Monitor/M32U#kf Case: lian-li.com/product/o11-air-mini/ Motherboard: www.gigabyte.com/Motherboard/B650-GAMING-X-AX-rev-10-11-12#...
Q&A December 2023
Просмотров 15 тыс.8 месяцев назад
Links from the stream: Atomic operator in Raku/Perl 6: andrewshitov.com/2019/09/09/atomic-operations-in-perl-6/ Helsing on Glassdoor: www.glassdoor.com/Reviews/Helsing-Reviews-E6763957.htm Simon Singh's The Code Book: simonsingh.net/books/the-code-book/ Rustc dev guide: rustc-dev-guide.rust-lang.org/getting-started.html I automated my bad keyboard: www.reddit.com/r/MechanicalKeyboards/comments/...
December 2023 Q&A
Просмотров 68Месяц назад
Also available as a video on RUclips (ruclips.net/video/A3qBmLx9uTU/видео.html) . Questions and answers session from December, 2023. Link list • Atomic operator in Raku/Perl 6: andrewshitov.com/2019/09/09/atomic-operations-in-perl-6/ • Helsing on Glassdoor: www.glassdoor.com/Reviews/Helsing-Reviews-E6763957.htm • Simon Singh’s The Code Book: simonsingh.net/books/the-code-book/ • Rustc dev guide...
Making the (partial) Rust BitTorrent client more reasonable
Просмотров 35 тыс.9 месяцев назад
After doing the BitTorrent CodeCrafters challenge in #rustlang (ruclips.net/video/jf_ddGnum_4/видео.html), I had an urge to do some re-organization to better fit what a real implementation would look like when it has to handle files, multiple peers, multiple pieces, concurrency, etc. So, that's what we did! If you want to give the whole challenge a try, please consider using my refrral link: ap...
Implementing (part of) a BitTorrent client in Rust
Просмотров 91 тыс.10 месяцев назад
In this stream, we're doing the "implement BitTorrent" challenge from app.codecrafters.io/join?via=jonhoo in Rust. Essentially, we're implementing a BitTorrent client from scratch by following a test-guided set of steps. This is a way of learning I really like (and encourage others to try), and after this stream I'm comfortable saying that this is also a good _implementation_ of that learning m...
Open Source Maintenance, 2023-08-25
Просмотров 14 тыс.Год назад
I'm nearly back up speed with my GitHub notifications across the various open-source projects I maintain since my move. In this stream, after many more like it, we finally make it (more or less) to the end of those notifications! As with the past ones of these, my hope is that by showing the maintainer side of open-source, you get a better sense for how open-source operates, what it's like to b...
Open Source Maintenance, 2023-08-13
Просмотров 9 тыс.Год назад
Open Source Maintenance, 2023-08-13
Open Source Maintenance, 2023-08-06
Просмотров 13 тыс.Год назад
Open Source Maintenance, 2023-08-06
Decrusting the axum crate
Просмотров 68 тыс.Год назад
Decrusting the axum crate
Q&A July 2023
Просмотров 21 тыс.Год назад
Q&A July 2023
July 2023 Q&A
Просмотров 27Месяц назад
July 2023 Q&A
Crust of Rust: std::collections
Просмотров 70 тыс.Год назад
Crust of Rust: std::collections
Supply Chain Security - MIT 6.5660 Computer Security guest lecture
Просмотров 16 тыс.Год назад
Supply Chain Security - MIT 6.5660 Computer Security guest lecture
Solving distributed systems challenges in Rust
Просмотров 234 тыс.Год назад
Solving distributed systems challenges in Rust
Setting up CI and property testing for a Rust crate
Просмотров 26 тыс.Год назад
Setting up CI and property testing for a Rust crate
From cargo to crates.io and back again
Просмотров 20 тыс.Год назад
From cargo to crates.io and back again
Making a Presentation: Living with Rust long-term
Просмотров 36 тыс.Год назад
Making a Presentation: Living with Rust long-term
Decrusting the serde crate
Просмотров 56 тыс.Год назад
Decrusting the serde crate
Q&A December 2022
Просмотров 15 тыс.Год назад
Q&A December 2022
December 2022 Q&A
Просмотров 16Месяц назад
December 2022 Q&A
Q&A November 2022
Просмотров 13 тыс.Год назад
Q&A November 2022
November 2022 Q&A
Просмотров 148Месяц назад
November 2022 Q&A
Day in the Life of Open Source Maintenance: 2022-10-08
Просмотров 25 тыс.Год назад
Day in the Life of Open Source Maintenance: 2022-10-08

Комментарии

  • @OccultDemonCassette
    @OccultDemonCassette 23 часа назад

    Okay what is going on in Japan and why are so many Tokyo workers running around grabbing futures. I just woke up and I'm half asleep and RUclips autoplay this and I do not understand anything that is being said. Who is waking up Japanese poll workers and why?

  • @JG-nm9zk
    @JG-nm9zk День назад

    This is one crusty corner of rust.

  • @daniellarson6279
    @daniellarson6279 День назад

    The reason that the book includes the original string in every token is basically only for error reporting. With miette you handle all of the necessary state to track that during iteration, so it would probably be reasonable to only save anything for idents, numbers, and strings. With that you could keep everything in TokenKind. Actually on second thought, that doesn't really work until you get to the compile-as-you go bytecode interpreter, you lose your original place with runtime errors in the tree-walk interpreter. It could be an interesting exercise in how to make it more "rusty" if you get to the bytecode section, I guess.

  • @Lado93
    @Lado93 День назад

    1:20:00 this hack to satisfy compiler are pretty crazy

  • @Hellbending
    @Hellbending 2 дня назад

    20:27 - Cat confirmed employed. Better be paying a decent wage man, they be hungry out here on a budget 😂😂😂❤

  • @bullxtraders1366
    @bullxtraders1366 2 дня назад

    Subscribe, like & share🎉

  • @mahskasramot7468
    @mahskasramot7468 2 дня назад

    1:15:36

  • @Lino-ty5hv
    @Lino-ty5hv 3 дня назад

    Around 5:00:00; parsing things like ‘for’ and function calls as a special type of operator reminds me of a more general idea from some functional programming languages - particularly interactive theorem provers like Agda and Coq - called *mixfix* operators. In that setting, you can have an operator like ‘if_then_else_’ which is a prefix operator with two ‘internal arguments’, with the positions of the arguments specified by the _’s. This isn’t something that can be handled by basic Pratt parsing though, and most of the implementations out there go the extra step of allowing user defined operators too, so it’s definitely overkill for this project!

  • @inf0phreak
    @inf0phreak 4 дня назад

    Regarding the first Q&A part (about 20 minutes in), the way I'd explain bytecode is that it's machine code for an ISA that doesn't physically exist, but instead is implemented in software, and that implementation can then be through e.g. an interpreter or a JIT compiler.

  • @svenyboyyt2304
    @svenyboyyt2304 4 дня назад

    Decrust ratatui please

  • @levonoganyan6183
    @levonoganyan6183 5 дней назад

    Another common case to use JoinSet is: when you have separate tasks doing their own thing, but want to kill all of them if one of the tasks errors out. E.g. you have some receiver task, which pushes data to some/multiple processor tasks and you have a signalling task, which waits for external commands and/or interrupts. If any of these tasks completes you want to be able to shut down the rest and maybe restart the whole thing

  • @clubpenguinfan1928
    @clubpenguinfan1928 7 дней назад

    I've been playing around with a parser style where the state machine is more explicit, e.g. with immutable state and functions LexerState -> Result<(Token, String), E>, and where you combine and compose those functions with monad language. Which is really cool, but then I took uncomfortably long to write out a sexp parser, and I decided I wanted to get back to the basics for a bit. Good old ~linear parsing, no big lookahead/backtracking, but still with good type safety, healthy memory idioms and good error reporting. Plus I haven't touched Rust in a while, so this video is the perfect excuse. Thanks for the videos as always, they rock -- hope we get a part 2 for this one!

    • @arialpew5248
      @arialpew5248 21 час назад

      What you are doing is called a parser combinator. It work and it's probably a standard to create your own parser theses days. Like most generic interface it can be tough to understand what's going on at some point. And painfull to maintain - especially when it come to error message. Bare function call with a side-effect parser is the opposite, like Jon did. They are dual to each others. I have no real preference tbh, both work equally well and have advantage/disadvantage. Using parser generator is also a solution, like yacc/bison.

    • @clubpenguinfan1928
      @clubpenguinfan1928 13 часов назад

      @@arialpew5248 I know, I've been looking into them recently. But yeah, I felt like I wanted to explore the other side too.

  • @Sebastian-vz2in
    @Sebastian-vz2in 8 дней назад

    3:14:26 i think we need ...or_else(|| self.rest.len() -1). Otherwise my code crashed, when the comment was the last line

  • @AmosRothberg
    @AmosRothberg 8 дней назад

    At 45:14 - 48:00 you talk about shutdown, and unless I misunderstood the documentation you make incorrect statements. for instance: when the main future exits, the OS does NOT immediately shut down all the threads - from the docs: 'Tasks spawned through Runtime::spawn keep running until they yield. Then they are dropped. They are not guaranteed to run to completion, but might do so if they do not yield until completion. Blocking functions spawned through Runtime::spawn_blocking keep running until they return. The thread initiating the shutdown blocks until all spawned work has been stopped. This can take an indefinite amount of time. The Drop implementation waits forever for this.' also, shutdown_background & shutdown_timeout do NOT shutdown gracefully.

  • @manojbabu3640
    @manojbabu3640 8 дней назад

    yeah. to switch text from upper to lower case or vice versa you can use gu or gU optionally you select using a motion and apply it on selection.

  • @carterthaxton
    @carterthaxton 9 дней назад

    Very cool! You inspired me to try the CodeCrafters challenge myself, and just completed the whole thing... Alas, the CodeCrafters site only goes up through basic statement evaluation with global variables, and doesn't cover the rest of the book. I suppose nothing is stopping me from continuing on myself, though. Thanks for doing this! It's fun to see how different our approaches were.

  • @alphabitserial
    @alphabitserial 9 дней назад

    Did you ever find out who was committing to your repo?

  • @DeathSugar
    @DeathSugar 9 дней назад

    Why don't you use some darkmode extensions?

    • @jonhoo
      @jonhoo 9 дней назад

      Because I like black text on white background for websites :)

  • @jRsqILVOY
    @jRsqILVOY 9 дней назад

    This was really fun, personally I'd love if you added more "best practices" / convenience crates like camino etc. as I feel a lot of the educational value comes from seeing how to do things well. E.g. all the work on the Iterator that uses references to the source string is much more useful to see than just cloning the string as you go character by character.

  • @stretch8390
    @stretch8390 9 дней назад

    A really small query: when Jon said, "... as you get characters from the input you produce tokens from the output...", at ~52:22 did he mean you produce tokens for the output? Just new to this and want to make sure I haven't missed something about how input/output interact.

    • @LtdJorge
      @LtdJorge 9 дней назад

      Yep. Tokens could be enum variants, references to the original input, etc. Maybe he meant “from” as a pipe side. Characters are taken _from_ the input and tokens are produced _from_ the output in an iterator.

  • @stretch8390
    @stretch8390 10 дней назад

    Just saw that you were one of the magicians who wrote The Missing Semester Course too. You have put out some incredibly useful resources.

  • @Benabik
    @Benabik 10 дней назад

    Pratt parsing for statements seems like huge overkill. Just match on the first item, if it's a keyword handle that particular form, if it's not, go to (Pratt) parsing expressions. Pratt parsing's win is handling all the nesting and recursion and you have none of that at the statement or block level. Notably, there is no binding power at the expression or block level, so you don't need to pass around r_bp everywhere.

  • @WatchingScience-gv8pe
    @WatchingScience-gv8pe 11 дней назад

    Python's CPython doesn't do JIT btw. Though it does have alternate interpreters that do JIT (pypy)

    • @ilyapopov823
      @ilyapopov823 10 дней назад

      Next version, 3.13 (to be released soon), has a JIT

    • @1vader
      @1vader 8 дней назад

      @@ilyapopov823 Though it looks like it will be disabled by default for now and currently doesn't do a whole lot of optimizations yet. But definitely opens up a lot of possible performance improvements.

  • @psi4j
    @psi4j 11 дней назад

    You're killing me with the light mode, Jon. Great content though.

  • @danygagnon8446
    @danygagnon8446 11 дней назад

    You are such an amazing teacher. Thanks so much for everything you have done.

  • @jithin.johnson
    @jithin.johnson 11 дней назад

    I love the 'Implementation' videos from you Jon, keep up the good work! 💪🏾

  • @okira_e
    @okira_e 11 дней назад

    Awesome content! This is what Tech RUclips needs more of.

  • @arjix8738
    @arjix8738 11 дней назад

    00:00 my eyes are toasted from that light theme

  • @jly_dev
    @jly_dev 11 дней назад

    Naruto whirlpools, Rasengan, I'm in 🚀

  • @Atharvak1910
    @Atharvak1910 11 дней назад

    My cat mauls me when i try to get it to switch to rust from C

  • @qaqkirby9781
    @qaqkirby9781 11 дней назад

    hi, jon, it seems like the parser isn't fully complete, will there be another stream about this?

    • @jonhoo
      @jonhoo 11 дней назад

      Yeah, I think this warrants a part two!

  • @luk17032000
    @luk17032000 11 дней назад

    The link to matklads article in the desription seems broken, I get a 404 on github pages. I think youtube parses the parenthesis as part of the link, the irony!

    • @jonhoo
      @jonhoo 11 дней назад

      RUclips's video descriptions are truly a mess 😅 How's it now?

    • @luk17032000
      @luk17032000 11 дней назад

      @@jonhoo It works now! Thanks :)

  • @SachinSharma590
    @SachinSharma590 11 дней назад

    Please make more such projects building in Rust with industry practices.

  • @bamless95
    @bamless95 11 дней назад

    Small correction: cpython (the python interpreter) is indeed a bytecode interpreter, so there is a compilation step that goes from AST to a bespoke bytecode instruction set. Old versions of ruby and perl are a better example for tree walking interpreters, as they run directly on the parsed AST (even though I believe modern versions also compile to bytecode for efficiency reasons)

  • @johanngambolputty5351
    @johanngambolputty5351 11 дней назад

    1:00:29 I always thought you couldn't have multicursor in nvim? Is this a plugin? I've been using multiline selection with :s/find/replace in these cases.

    • @fadseng
      @fadseng 11 дней назад

      It's a standard vim feature: type Ctrl-V -> select lines -> insert mode -> change the text -> Esc, after that the change is applied to all the selected lines.

    • @beyondcatastrophe_
      @beyondcatastrophe_ 11 дней назад

      He's not doing multicursor, he's just going into Visual mode. Visual mode allows you to write a column, not actual multicorsor (Ctrl+D)

    • @johanngambolputty5351
      @johanngambolputty5351 11 дней назад

      @@beyondcatastrophe_ Thank you both, I shall try this.

    • @stretch8390
      @stretch8390 9 дней назад

      Just got to this part and wondered the same.

  • @echoptic775
    @echoptic775 11 дней назад

    Why are you adding so much unnecessary boilerplate and being so fancy? One of the reasons I started liking zig more than rust is because I've seen in a lot of rust projects with crazy amounts of abstractions, unnecessary type wrappers and misdirections where I can barely follow the code.

    • @tinusgraglin2079
      @tinusgraglin2079 11 дней назад

      Which part? I missed the live stream, but looking at the resulting code at the github, everything seem straightforward.

    • @pmmeurcatpics
      @pmmeurcatpics 9 дней назад

      4 hours in, don't see what you were talking about... I mean I personally disliked the ginormous TokenKind - I thought it may've been useful to break it up (e.g. have the MaybeEqual tokens as a separate subtype, which in turn is a variant of TokenKind), but that would actually be an _additional_ level of abstraction that I though was missing. So what exactly are you talking about?

  • @johndisandonato
    @johndisandonato 11 дней назад

    Super cool, thank you for making this! Can't wait to get through the video. I took a crack at Crafting Interpreters in Rust earlier this year, it was a lot of fun. Now I'm setting an early new year resolution to build a Hindley-Milner, LLVM based language (in Rust, of course).

  • @ngwemo
    @ngwemo 11 дней назад

    Always learning more concepts about rust. Appreciate a lot from Kenya . Though I could suggest for a copy of your book 😊. I’ll appreciate a lot.

    • @Kabodanki
      @Kabodanki 11 дней назад

      I think this kind of book could be given for free in PDF after a couple of years. Money is hard to come by.

    • @kodesafi
      @kodesafi 11 дней назад

      Kenyans 😂. The web version of the book is free. Enda tu website.

  • @user-ql2jn1yu1h
    @user-ql2jn1yu1h 13 дней назад

    It was incredible useful, thanks very much!!

  • @CrapE_DM
    @CrapE_DM 14 дней назад

    I would describe Java enums like this. The enum declaration declares a type, and the options within an enum are instances of that type. So, each option has to have the same set of data.

  • @liz2k
    @liz2k 16 дней назад

    Thank you for that tutorial... Very helpful to understand all background of this :)

  • @sudomateo
    @sudomateo 18 дней назад

    Great content! Your thoughts on using sponsorship to create more content to be available for free resonates with me. I have the same principle for my content- I just haven't set up a sponsorship pipeline yet. If you're ever looking to collaborate on content or just chat technology happy to do so! Subscribed for more.

  • @stretch8390
    @stretch8390 19 дней назад

    Is anyone able to tell me whether anything has changed with async/await in rust since this video? Just wanting to confirm before committing the time

    • @josephsmith5110
      @josephsmith5110 16 дней назад

      async fn in traits is now allowed and RFCs for async closures and naming the return type of (async) methods have been accepted.

  • @PiyushKumbhare10
    @PiyushKumbhare10 22 дня назад

    Pretty new to Rust and don't really have any async/await requirements or anything, but I've always wondered how it worked under the hood. This was the perfect video for learning, thanks so much 😄