finished ch17.6 and ch18 intro

.github/workflows/rust.yml

@@ -91,7 +91,7 @@ jobs:
 
     # Step 2: Run unit and integration tests (excluding documentation tests)
     - name: Run Tests
-      run: cargo test --tests --verbose
+      run: cd minigrep/ && cargo test --tests --verbose
 
   # name of the job
   documentation-check:
@@ -121,6 +121,7 @@ jobs:
     # Step 3: Check if documentation tests were run
     - name: Check for Documentation Tests
       run: |
+        cd minigrep/ &&
         DOC_TESTS=$(cargo test --doc --verbose)
         if [[ ! "$DOC_TESTS" =~ "running" ]]; then
           echo "No documentation tests were run!" && exit 1

.obsidian/workspace.json

@@ -91,6 +91,48 @@
               "title": "Traits for Async"
             }
           },
+          {
+            "id": "c00c13dd25b12ad4",
+            "type": "leaf",
+            "state": {
+              "type": "markdown",
+              "state": {
+                "file": "Futures, Tasks and Threads Together.md",
+                "mode": "source",
+                "source": false
+              },
+              "icon": "lucide-file",
+              "title": "Futures, Tasks and Threads Together"
+            }
+          },
+          {
+            "id": "e8a505fdeccc0275",
+            "type": "leaf",
+            "state": {
+              "type": "markdown",
+              "state": {
+                "file": "OOP Programming Features.md",
+                "mode": "source",
+                "source": false
+              },
+              "icon": "lucide-file",
+              "title": "OOP Programming Features"
+            }
+          },
+          {
+            "id": "b49e674e0ebaaeb7",
+            "type": "leaf",
+            "state": {
+              "type": "markdown",
+              "state": {
+                "file": "Characteristics of OO Languages.md",
+                "mode": "source",
+                "source": false
+              },
+              "icon": "lucide-file",
+              "title": "Characteristics of OO Languages"
+            }
+          },
           {
             "id": "2a974ca5442d705f",
             "type": "leaf",
@@ -144,7 +186,7 @@
             }
           }
         ],
-        "currentTab": 5
+        "currentTab": 8
       }
     ],
     "direction": "vertical"
@@ -287,10 +329,13 @@
       "command-palette:Open command palette": false
     }
   },
-  "active": "ee4116419493acd3",
+  "active": "b49e674e0ebaaeb7",
   "lastOpenFiles": [
-    "Futures in Sequence.md",
+    "OOP Programming Features.md",
+    "Characteristics of OO Languages.md",
+    "Futures, Tasks and Threads Together.md",
     "Traits for Async.md",
+    "Futures in Sequence.md",
     "Any Number of Futures.md",
     "Futures and Async.md",
     "Async, Await, Futures and Streams.md",
@@ -313,9 +358,6 @@
     "Project Organization.md",
     "Writing_Tests.md",
     "minigrep/src/lib.rs",
-    "Test_Organization.md",
-    "Traits.md",
-    "Modules and Use.md",
     "does_not_compile.svg",
     "Untitled.canvas",
     "Good and Bad Code/Commenting Pratices",

Characteristics of OO Languages.md

@@ -0,0 +1 @@
+# Characteristics of Object-Oriented Languages

Futures, Tasks and Threads Together.md

@@ -0,0 +1,157 @@
+# Putting It All Together: Futures, Tasks and Threads
+As we saw [here](./Concurrency.md), threads provide one approach to concurrency.
+
+Another approach was using async with futures and streams.
+
+If you were wondering when to choose one over the other.
+
+The answer is that it depends, and in many cases, the choice isn't threads *or* async but rather threads *and* async.
+
+Many OS have supplied threading-based concurrency models for decades now, and many programming results support them as a result.
+
+These models are not without their own tradeoffs.
+
+On many OSes, they use a fair bit of memory, and they come with some overhead for starting up and shutting down.
+
+Threads are also only an option when your OS and hardware support them.
+
+Unlike a modern desktop and mobile computer, some embedded systems don't have an OS at all, so they also don't have threads.
+
+The async model provides a different and complementary set of tradeoffs.
+
+In the async model, concurrent operations don't require their own threads.
+
+Instead, they can run on tasks (just like when we used `trpl::spawn_task`)m this kicks off work form a synchronous function in the streams section.
+
+A task is similar to a thread, instead of being managed by the OS, it is managed by library-level code: the runtime
+
+Previously, we saw that we could build a stream by using async channel and spawning an async task we could call from synchronous code.
+
+We then can do this exact same thing with a thread.
+
+Before we used `trpl::spawn_task` and `trpl::sleep`.
+
+Here we replaced those with the `thread::spawn` and `thread::sleep` APIs from the std library in the `get_intervals` function.
+```rust
+fn get_intervals() -> impl Stream<Item = u32> {
+    let (tx, rx) = trpl::channel();
+
+    // This is *not* `trpl::spawn` but `std::thread::spawn`!
+    thread::spawn(move || {
+        let mut count = 0;
+        loop {
+            // Likewise, this is *not* `trpl::sleep` but `std::thread::sleep`!
+            thread::sleep(Duration::from_millis(1));
+            count += 1;
+
+            if let Err(send_error) = tx.send(count) {
+                eprintln!("Could not send interval {count}: {send_error}");
+                break;
+            };
+        }
+    });
+
+    ReceiverStream::new(rx)
+}
+```
+If you run this code it will produce an identical output as the one before.
+
+And notice how little changes here from the perspective of calling code.
+
+What is more even though one of our functions spawned an async task on the runtime and the other spawned an OS thread.
+
+The resulting streams were unaffected by the differences.
+
+Despite their similarities, these two behave very differently, although we might have a hard time measuring it in this very simple example.
+
+Alternatively we could spawn millions of async tasks on any modern personal computer.
+
+If we tried to do that with threads, we would literally run out of memory.
+
+There is a reason that these APIs are so similar.
+
+Threads act as a boundary for sets of synchronous operations; concurrency is possible *between* threads.
+
+Tasks act as a boundary for sets of *asynchronous* operations.
+
+Concurrency is possible both *between* and *within* tasks, because a task can switch between futures in its body.
+
+Finally, futures are Rust's most granular unit of concurrency, and each future may represent a tree of other futures.
+
+The runtime (specifically, its executor) manages tasks and tasks manage futures.
+
+In this regard, tasks are similar to lightweight, runtime-managed threads with added capabilities that come from being managed by a runtime instead of by the operating system.
+
+This doesn't mean that async tasks are always better than threads (or vice versa).
+
+Concurrency with threads is in some ways a simpler programming model than concurrency with `async`.
+
+This can be either a strength or a weakness.
+
+Threads are somewhat "fire and forget".
+
+They have no native equivalent to a future, so they simply run to completion without being interrupted except by the OS itself.
+
+That is they have no built-in support for *intratask concurrency* the way futures do.
+
+Threads in Rust also have no mechanisms for cancellation (we haven't covered explicitly in this ch but was implied by the fact that whenever we ended a future, tis state got cleaned up correctly).
+
+The limitations also make threads harder to compose than futures.
+
+This is much more difficult.
+
+For example, to use threads to build helpers such as the `timeout` and `throttle` methods that we built earlier.
+
+The fact that futures are richer data structures means they can be composed together more naturally as we have seen.
+
+Tasks, give us *additional* control over futures, allowing us to choose where and how to group them.
+
+It turns out that threads and tasks often work very well together, because tasks (in some runtimes) can be moved around between threads.
+
+In fact, under the hood, the runtime we have been using including the `spawn_blocking` and `spawn_task` functions is multithreaded by default.
+
+Many runtimes use an approach called *work stealing* to transparently move tasks around between threads.
+
+Based on how the threads are currently being utilized, to improve the system's overall performance.
+
+This approach actually requires threads *and* tasks and therefore futures.
+
+When thinking about which method to use when, consider these rules of thumb:
+- If the work is *very parallelizable*, such as processing a bunch of data where each part cab be processed separately, threads are a better choice.
+- If the work is *very concurrent*, such as handling message from a bunch of different sources that many come in at different intervals or different rates, async is a better choice.
+If you need both concurrency and parallelism, you don't have to choose between threads and async.
+
+You can use them together freely, letting each one play the part it is best at.
+
+An example of this, below, shows a fairly common example of this kind of mix in real-world Rust code.
+```rust
+use std::{thread, time::Duration};
+
+fn main() {
+    let (tx, mut rx) = trpl::channel();
+
+    thread::spawn(move || {
+        for i in 1..11 {
+            tx.send(i).unwrap();
+            thread::sleep(Duration::from_secs(1));
+        }
+    });
+
+    trpl::run(async {
+        while let Some(message) = rx.recv().await {
+            println!("{message}");
+        }
+    });
+}
+```
+Here we being by creating an async channel, then spawn a thread that takes ownership of the sender side of the channel.
+
+Within the thread, we send the numbers 1-10, sleeping for a second between each.
+
+Finally, we run a future created with an async block passed to `trpl::run` just as we have throughout the chapter.
+
+In this future, we await those messages, just as in the other message-passing examples we have seen.
+
+Returning to the scenario we opened the chapter with, imagine running a set of video encoding tasks using a dedicated thread (because video encoding is compute-bound) but notifying the UI that those operations are dont with an async channel.
+
+There are countless examples of these kinds of combinations in real-world use cases.

OOP Programming Features.md

@@ -0,0 +1,16 @@
+# Object-Oriented Programming Features of Rust
+OOP is a way of modeling programs.
+
+Objects as a programmatic concept were introduced in the programming language Simula in the 1960s.
+
+These objects influenced Alan Kay's programming architecture in which objects pass messages to each other.
+
+To describe this architecture, he coined the term *object-oriented programming* in 1967.
+
+Many competing definitions describe what OOP is, and by some of these definitions Rust is object-oriented, but by other it is not.
+
+In this section, we will explore certain characteristics that are commonly considered object-oriented and how those characteristics translate to idiomatic Rust.
+
+Next we will show how to implement an object-oriented design pattern in Rust and discuss the trade-offs of doing so.
+
+Versus implementing a solution using some of Rust's strengths instead.

Traits for Async.md

@@ -201,3 +201,205 @@ This was in terms of `Unpin`, not `Pin`.
 
 How does `Pin` relate to `Unpin` and why does `Future` need `self` to be in a `Pin` type to call `poll`?
 
+Remember from before, a series of await points in a future get compiled into a state machine, and the compiler makes sure that state machine follows all of Rust's normal rules around safety, which includes borrowing and ownership.
+
+In order to make this work, Rust looks at what data is needed between one await point and either the next await point or the end of the async block.
+
+Each variant get the access it needs to the data that will be used in that section of the source code, whether by taking ownership of that data or by getting a mutable or immutable reference to it.
+
+If we get anything wrong about the ownership or references in a given async block, the borrow checker will tell us.
+
+When we want to move around the future that corresponds to that block, like moving it into a `Vec` to pass to `join_all`, where things get tricker.
+
+When we move a future, either by pushing it into a data structure to use as an iterator with `join_all` or by returning from a function, this actually means moving the state machine Rust creates for us.
+
+Unlike most other types in Rust, the future Rust creates for async blocks can end up with references to themselves in the fields of any given variant.
+
+This is shown in this illustration
+<img src="https://doc.rust-lang.org/book/img/trpl17-04.svg" />
+By default, any object that has a reference to itself is unsafe to move, because references always point to the actual memory address of whatever they refer to.
+
+If you move the data structure itself, those internal references will be left pointing to the old location.
+
+However that memory location is now invalid.
+
+One thing is that its value will not be updated when you make changes to the data structure.
+
+Another thing, which is more important, is the computer is now free to reuse that memory for other purposes.
+
+You could end up reading completely unrelated data later.
+
+<img src="https://doc.rust-lang.org/book/img/trpl17-05.svg" />
+Theoretically, the Rust compiler could try to update every reference to an object whenever it gets moved, but that could add a lot of performance overhead.
+
+This is especially true if a whole web of references needs updating.
+
+If we could instead ensure that the data structure *doesn't move in memory*, we then wouldn't have to update any references.
+
+This is exactly what Rust's borrow checker requires: in safe code, it prevents you from moving any item with an active reference to it.
+
+`Pin` builds on that give us the exact guarantee we need.
+
+When we *pin* a value by wrapping a pointer to that value in `Pin`, it can no longer move.
+
+Thus if you have `Pin<Box<SomeType>>`, you actually pin the `SomeType` value, *not* the `Box` pointer.
+
+The image illustrates this process.
+<img src="https://doc.rust-lang.org/book/img/trpl17-06.svg" />
+In fact, the `Box` pointer can still move around freely.
+
+We car about making sure the data ultimately being referenced stays in place.
+
+If a pointer moves around, *but the data it points is in the same place*, there is no potential problem.
+
+As an independent exercise, look at the dos for the types as well as the `std::pin` module and try to work out how you would do do this with a `Pin` wrapping a `Box`.
+
+The key is that the self-referential type cannot move, because it is still pinned.
+<img src="https://doc.rust-lang.org/book/img/trpl17-07.svg" />
+However most types are perfectly safe to move around, even if they happen to be behind a `Pin` pointer.
+
+We only need to think about pinning when the items have internal references.
+
+Primitives values such as numbers and Booleans are safe since they obviously don't have any internal references, so they are obviously safe.
+
+Neither do most types you normally work with in Rust.
+
+You can move around a `Vec`, for example, without worrying.
+
+given what we have seen, if you have a `Pin<Vec<String>>`, you would have to everything via the safe but restrictive APIs provided by `Pin`/
+
+Even though a `Vec<String>` is always safe to move if there are no other references to it.
+
+We need a way to tell the compiler that it is fine to move items around in cases like this, this is where `Unpin` comes into action.
+
+`Unpin` is a marker trait, similar to the `Send` and `Sync` traits.
+
+Thus has no functionality of its own.
+
+Marker traits exist only to tell the compiler to use the type implementing a given trait in a particular context.
+
+`Unpin` informs the compiler that a given type does *not* need to uphold any guarantees about whether the value in question can be safely moved.
+
+Just like `Send` and `Sync`, the compiler implements `Unpin` automatically for all types where it can prove it is safe.
+
+A special case, is where `Unpin` is *not* implemented for a type.
+
+The notation for this is `impl !Unpin for *SomeType*`, where `*SomeType*` is the name of a type that *does* need to uphold those guarantees to be safe whenever a pointer to that type is used in a `Pin`.
+
+The relationship between `Pin` and `Unpin` has two important things to remember:
+- `Unpin` is the "normal case", `!Unpin` is the special case
+- Whether a type implements `Unpin` or `!Unpin` *only* matters when you are using a pinned pointer to that type like `Pin<&mut *SomeType*>`
+
+To make that concrete, think about a `String`: it has a length and the Unicode characters that make it up.
+
+We can wrap a `String` in `Pin`.
+
+However `String` automatically implements `Unpin` as do most other types in Rust.
+<img src="https://doc.rust-lang.org/book/img/trpl17-08.svg" />
+	Pinning a `String`; the dotted line indicates that the `String` implements the `Unpin` trait, and thus is not pinned.
+This results, in the ability to do things that would be illegal if `String` implemented `!Unpin`, such as replacing one string with another at the exact same location in has no interval references that make it unsafe to move around.
+
+This wouldn't violate the `Pin` contract, because `String` has no internal references that make it unsafe to move around.
+
+This is precisely why it implements `Unpin` rather than `!Unpin`.
+<img src="https://doc.rust-lang.org/book/img/trpl17-09.svg" />
+Now that we know enough to understand the errors reported for that `join_all` call from before.
+
+There we originally tried to move the futures produced by the async blocks into a `Vec<Box<dyn Future<Output = ()>>>`.
+
+As we have seen, those futures may have internal references, so they don't implement `Unpin`.
+
+They need to be pinned and then we can pass the `Pin` type into the `Vec`, confident that the underlying data in the futures will *not* be moved.
+
+`Pin` and `Unpin` are mostly important for building lower-level libraries, or when you are building a runtime itself, rather than for day-to-day Rust.
+
+When you see these traits in error messages, now you will have a better idea of how to fix your code.
+
+Note, the combination of `Pin` and `Unpin` makes it possible to safely implement a whole class of complex types in Rust that would otherwise prove challenging because they are self-referential.
+
+Types that require `Pin` show up most commonly in async Rust today.
+
+Every once in a while, you may see them in other contexts too.
+
+The specifics of how `Pin` and `Unpin` work, and the rules they are required to uphold are covered extensively in the `API` documentation for `std::pin`, so you can check there for more info.
+
+In fact there is a whole BOOK on async Rust programming, that you can find [here](https://rust-lang.github.io/async-book/)
+
+## The `Stream` Trait
+As we leaned earlier, streams are similar to asynchronous iterators.
+
+Unlike `Iterator` and `Future`, `Stream` has no definition in the std library (as of writing this), but there *is* a very common definition form the `fuitures` crate used throughout the ecosystem.
+
+Here is a review of the `Iterator` and `Future` traits before going into how a `Stream` trait might merge them together.
+
+From `Iterator`, we have the idea of a sequence: its `next` method provides an `Option<Self::Item>`
+
+From `Future`, we have the idea of readiness over time: the `poll` method provides a `Poll<Self::Output>`
+
+This allows us to represent a sequence of items that become ready over time, we define a `Stream` trait that puts those features together.
+```rust
+use std::pin::Pin;
+use std::task::{Context, Poll};
+
+trait Stream {
+    type Item;
+
+    fn poll_next(
+        self: Pin<&mut Self>,
+        cx: &mut Context<'_>
+    ) -> Poll<Option<Self::Item>>;
+}
+```
+Here the `Stream` trait defines an associated type called `Item` for the type of the items produced by stream.
+
+This is similar to `Iterator`, where there may be zero to many items, and unlike `Future`, where there is always a single `Output`, even if it is the unit type `()`.
+
+`Stream` also defines a method to get those items.
+
+We call it `poll_next`, to make it clear that it polls in the same way `Future::poll` does and produces a sequence of items in the same way `Iterator::next` does.
+
+Its return type combines `Poll` with `Option`.
+
+
+The outer type is `Poll`, because it has to be checked for readiness, just as a future does.
+
+The inner type is `Option`, because it needs to signal whether there are more messages, just as an iterator does.
+
+Somethin like this will likely end up as part of Rust's standard library.
+
+In the meantime, it is part of the toolkit of most runtimes, so you can rely on it, and everything that will be covered should apply generally.
+
+In the example we saw previously in the section on streaming, we didn't use `Poll_next` or `Stream`, but instead used `next` and `StreamExt`.
+
+We *could* work with futures directly via their `poll` method.
+
+Using `await` is much nicer, and the `StreamExt` trait supplies the `next` method so we can do just this:
+```rust
+trait StreamExt: Stream {
+    async fn next(&mut self) -> Option<Self::Item>
+    where
+        Self: Unpin;
+
+    // other methods...
+}
+```
+Note: The definition that we used earlier in the ch looks slightly different that this.
+
+This is because it supports versions of Rust that did not yet support using async functions in traits.
+
+As a result it looks like this:
+```rust
+fn next(&mut self) -> Next<'_, Self> where Self: Unpin;
+```
+This `Next` type is a `struct` that implements `Future` and allows us to name the lifetime of the reference to `self` with `Next<'_, Self>`, so that `await` can work with this method.
+
+The `StreamExt` trait also has some interesting method available to use with steams.
+
+`StreamExt` is automatically implemented for every type that implements `Stream`.
+
+These traits are defined separately to enable the community to iterate on convenience APIs without affecting the foundational trait.
+
+In the version of `StreamExt` used in the `trpl` crate, the trait not only defines the `next` method but also supplies a default implementation of `next` that correctly handles the details of calling `Stream::poll_next`.
+
+Meaning that even when you need to write your own streaming data type, you *only* have to implement `Stream` and then anyone who uses your data type can use `StreamExt` and its methods with it automatically.
+