斯坦福 CS110 计算机系统原理笔记（七）
P20：Lecture 19 Principles of System Design - main - BV1ED4y1R7RJ
So， CS140 is about operating systems。 In this class。
we used all these things that you will end up building in operating systems， right？
Like you build a file system。 You build like how you actually。
you kind of build a little bit of that for this class。
but you build like a little lower level file system and more robust and more of the parts。
for the files。 You don't have to just read and write files。 You build the actual file system in 140。
You build a threading library or a forking library and multi-processing library。
And those are the kinds of things that we know how to use now and then you'll actually。
do them in 140 where you actually build them and understand what it takes to build a threading。
library。 So， it's a good thing you already know how to use these because then it's much easier。
to understand how you're going to go and build them。 So， that's that。
And that's about the implementation part of it。 There's also CS 143 which is compiler construction。
Compilers class， when I was in graduate school， it was one of my favorite classes。
I didn't do particularly well， I'm sorry。 But it was a class that I enjoyed a lot because you really get a much more detailed feel for。
hey， how does your code end up as machine code？ Now。
you cover that a little bit in 107 but not really the part of like actually doing， the translation。
Like you see how， you see what the beginning and the end are but compiler is all about。
translating and doing that。 And it's a great class。
I also happen to think that a lot of employers when they see that you took a compiler's class。
actually perk up a little bit and go， oh， well this person went a little bit outside。
than normal because it's not like everybody takes compilers and they know it's a hard， class too。
So， for what that's worth。 I think it's an interesting class and you might want to take a look at that。
If you like the networking things that we did recently， CS144 covers networking and this。
is in much more detail than what we did。 We just talked about server client and setting up a network and so forth。
This networking class digs into more of the details。
Like how do the bits actually get across the network efficiently？
We'll talk a little bit more about that today just in general terms。
But that's what the networking class covers and it's another good class especially if。
you're interested in this sort of stuff。 If you do go into security or things。
it's probably not a bad idea to take a peer networking。
class as well because it's got a lot of the things that a lot of the exploits end up exploiting。
things that might be at the networking layers that you haven't necessarily seen in this class。 Okay？
So， that's the three bigger ones that I think you could go on and take。
There's also 144 and 142 and other classes that are， sorry， 142 is another one that you。
can take that dig into more systems details。 So there are a lot of systems classes and again。
I do recognize that many of you are like， I'm glad I'm done with systems forever and that's it。
But for other people who want to do it， lots of other choices and feel free to chat with。
me about that if you want to know some of those other choices。 Okay。
So here's the principles we want to talk about today。 So we want to talk about abstraction。 Now。
abstraction is what you've been doing。 You've seen abstraction at many。
many different layers and we'll get to the details of what， abstraction is all about。
We need to talk about how that relates to CS110。 Modularity and layering is another one that we're going to look at。
Naming， remember all those things， you're doing file systems and like name resolution。
That's kind of a big principle that you have to deal with。
Many of these principles are all about doing things either more efficiently or doing them。
in such a way that makes a big system become less gigantic to wrap your head around。
That's what a lot of these things are。 Okay。 Or it's a matter of like name resolution is all about。
hey， humans are good at names， computers are good at numbers。
how do you mix those together so that it works most efficiently， for the computer。 Virtualization。
we'll talk about two different types of virtualization and then concurrency。
which I would argue this class is more about concurrency than probably any other topic。
There's more of that kind of filtered in along the way。
And then client server request and response is another one。
It's not just networks that deal with client server request and response。
Anytime you have two programs trying to communicate whether they're on the same computer or across。
a network or in some other sense， that's where you've got this client server request response。
And understanding how that works is kind of is important。 Okay。 All right。
So those are the let's jump right in and of course， hang on。 Of course， this is not working again。
There we go。 Okay。 Maybe。 Okay。 We'll go about it。
There we go。 There we go。 There we go。 There we go。
Now it's back。 Okay。 So abstraction。 All right。 Abstraction is simply separating what the program is doing from the implementation。
Okay。 And this is something we've seen in lots of different scenarios。
But one example is just a sorting program。 Right。 A sorting program has an interface where you pass a bunch of words in and they come back。
to you sorted。 And that's the interface that you may have。
You have to put them in an array and you have to set where the beginning and the array。
is and so forth。 That would be the interface。 The actual behavior is they get sorted。
How that implementation happens is completely up to the program。 Now， of course， with sort。
you want to do it。 You don't， there are so many different types of sorts and you don't want to do the one。
that's going to be oh and you want to do the one that's going to be oh and log and or， whatever。
But in terms of the implementation versus the or the interface versus the abstracted。
this is how it goes， that's the big， big picture of abstraction。 Okay。 We've seen these in many。
many ways。 The big challenge a lot of times is to design a clean interface that makes some library。
you're designing easy to use。 Okay。 We've actually done a lot of the designing of the interface for you in this class。
Mainly because it would be really hard to say hey， here's the， it would be hard to test。
it first of all。 But second， without a clear interface， you can spend many。
many hours thinking about how， to design an interface for something and you kind of want to do it properly because it。
is challenging。 I would argue that the C++ language， the interfaces are not necessarily as clean as。
you might hope for other languages。 They tend to be somewhat cleaner。 You know。
when you do a find in a map and you come and give you back an iterator and there's。
a first and a second and all that kind of business that's kind of painful to deal with。 All right。
That might not be quite as clean。 It's there for a reason， but it might not be particularly clean。
File systems。 So whenever， when you use file star and IO streams and we use many more IO streams in。
this class than we did file stars， that's an abstraction that is away from the read， the。
low level read and write， which is what we saw right at the beginning。
And why do we abstract those away？ Because again， it's easier。 With the read and the write。
you have to worry about buffering yourself。 You have to worry about， oh。
what happens if I try to read 10 bytes and I only get 6， bytes back。
These are all abstract in a way。 Once you have a good file library like IO stream and file stars and so forth。
Okay。 Processes。 Well， processes。 Why is that an abstraction？ Well。 Okay。 So it's basically。
you probably don't know how fork and exec VP work。 Right？
You'll learn that if you take CS 140 about how those actually work under the hood。
But there's an interface there。 When you say fork。
you know that it creates two processes and you don't really care how， it happens。
But you know that now you've got two processes and you've got a child and a parent and so， forth。
And there's lots of other things happening to keep track of the child to be able to wait。
PID and so forth。 That's all in abstraction。 Signals is another one。 If you took CS 107E。
you may have， you definitely did interrupts which are basically signals and。
you know a little bit more about how those work。 But the idea that you've got a。
the kernel keeping track of a bunch of things that you， don't necessarily， you want them。
you want the kernel to keep track of it。 You don't really know how it's doing that。
You just know that when something happens， the kernel goes and tells your program calls。
a handler function or whatever。 That's basically a signal。 So that's abstracted away too。 Your。
the abstraction there is， oh okay， I've got a signal handler that's going to get。
called when something happens and how does that actually， when we camp that， it's all。
abstracted away from you。 Okay。 If you go take the operating systems。
you'll learn more about how that happens or maybe， some double E classes too or you deal with that。
Threads， same sort of thing。 Remember threads and processes are similar。
Their threads are kind of lightweight processes but it's the same idea。 You've got these threads。
How does it work？ We don't really know。 We know how to use them and we know how to use them efficiently。
That's that。 HTTP。 So this is what your last assignment。
the proxy was all about and you HTTP defines how things。
get sent across the network and the various messages that it gets and that's abstracted， away again。
You can， you generally with your web browser don't need to worry about get and post and。
that sort of thing。 Okay。 So that's abstraction。
Okay。 Modularity and layering。 So when you talk about modularity。
we're basically saying subdivide a larger system into， a bunch of smaller systems。
This is decomposition。 You've been doing this since day one of one or six A or one or six B and it's important。
One of my favorite examples of a large system that's complex is believe it or not a Xerox， machine。
So how do people use， everybody's used a Xerox machine before， right？ Well we have some in our。
in the CS department that are able to copy like hundreds of exams。
for like CS106B and CS106A and whatever and these machines are super duper complicated。 Right？
If you go and you think about what has to happen to make copies of pages in a certain， form。
it's pretty intense。 So for instance， okay。 Right？ So copy machines are supposed to like copy paper。
That's the big picture， right？ Purple paper。 There you go。
So it's a copy paper and that's what they do。 But what else do they do？ Well。
they handle double sided。 Right？ What does that actually mean？ Right？
It means you either have to have a way to write on both sides of the paper， which is not really。
what happens generally。 That would be kind of weird especially because gravity might come into play a little bit。
But you also have to， or you have to flip the paper over and that's a mechanical engineering。
challenge。 Right？ So you've got that in there。 These things sometimes staple。 Right？
Stapling involves getting the paper exactly right the place。 I mean， oh， by the way。
it also has collating。 Right？ Which is where you have to have like all the different stacks of the right thing。
How does collating work？ Well， if you have a 20 page exam。
like the CS110 exam might not be 20 pages， but let's， say you have a 20 page exam， right？ Well。
they better be enough memory in your system to hold all 20 pages or some of the。
older machines had to refeed the paper every time it wanted to do a 20 page thing because。
they didn't have enough memory for that。 So that's a big thing because there's memory in there。
Let's see。 It's got networking associated with it if you can， like I can send a print job to the。
Xerox copy or that's it。 That's I can do that。 What else does it have？
There's also the whole physical layer or the actual physics that goes into it。 Like。
you guys know how copyers actually work， like physically how they do it？ Okay。
so there's a piece of paper that's black and oh by the way， there's also lasers， right？
There's a laser that's inside your copy machine， right？ And that's kind of cool。
And actually the laser charges up your paper。 Let's say it's a negative charge。
I don't know which charge。 It charges up the various dots on your paper where there's going to be a toner。
And then the rest of the copy machine takes the toner which is that powdery black horrible。
stuff and it charges at the opposite charge。 And then it dumps a bunch of toner on the paper believe it or not。
And then it where it charges are there it sticks to it。 And then it sweeps off to excess， all right？
Actually， I think first it fuses it and then it sweeps off or maybe the other way around。
But it basically does that and then it sort of says there's a heating business in there。
and that's why it always comes out in one。 And my point is that these things are so complex that no one person in fact no ten people know。
enough about the entire machine to be able to like fix it or do it all by themselves。
Or even understand it to be able to tell you all about all the details， right？ There's physicists。
there's engineers， there's oh by the way， there's all the software。
You know how many different menus are on that thing， right？ There's so much software on there。
right？ Like has to do it all and get everything right and then there's mechanical engineering that。
goes into it。 I mean there's so many people that work on that。 It has to be subdivided。
It's just way too big。 You would never have one person creating a Xerox machine by herself， right？
It's impossible because one person can't do that。 So that's that。
Another good example is the phone in your pocket， right？ If you took 107E with me。
you remember we talked about like all the different sensors， that are on your phone， right？
There's probably 15 different sensors on here。 Do you know there's four radios in your phone？
There's four different radios in your phone。 There's the one for the regular phone calls which isn't even the regular thing you do anymore。
That's like the least used thing you use your phone for anymore。 There's the Wi-Fi one。
There's a Bluetooth one。 There's， let's see， there's another one that communicates with the。
when you do Apple Pay， or whatever。 So I mean there's all sorts of radios in your phone。
There's cameras。 There's at least three different cameras。 There's a camera on the front。
There's a camera on the back。 There's a camera underneath the fingerprint thing。 If you've got that。
that's how that works， right？ Some ones have cameras built into the screen now。
I mean that's not even to mention there's a temperature sensor。 There's a drop sensor。
There's an accelerometer。 There's a gyroscope， right？ Who knows， understands all those things？
Nobody understands every little bit about that， right？
There's specific engineers whose job is to do the gyroscope part of the iPhone and that's。
how it works， right？ But it's all about subdivision of this large system。
So you have to understand how that works if you're going to be building bigger projects， right？
Now at the same time you might go to Apple and work on the iPhone and be like， "Oh， I'm。
the person who's doing the fingerprint sensor and whatever。"， And you're not the only one。
But you might feel like you're doing a very small part of a much bigger thing。
You have to keep that in mind that， "Oh， there is a bigger project here that ends up， being there。"。
But that's where you have to understand。 This class hopefully gave you assignments that were somewhat robust and there were lots。
of moving parts and that's why there's 20 different header files and whatever because。
you have to kind of try to understand that。 And we don't make it too complex that you can't as one person understand it but we need to。
push that limit to say， "Oh， we better break these into various subsystems otherwise it's。
not going to happen。"， Final systems。 Here's a good example of layering here where you've got lots of not necessarily modulating。
but you've got layering which is putting one layer on top of the other so that you break。
it down into its various components。 Remember for your assignment two there was this symbolic link where you didn't have to。
use symbolic links but you could read about those and learn about those。
There's an absolute path name which is like slash user slash class etc。
There's a path name layer which actually takes the which can take things like dot dot slash。
dot dot slash whatever and then layers it onto an actual thing there。
There's the file name layer which actually does the files for human reading that's where。
you've got that layer。 Then you've got the i-node layer。 Remember each file has an i-node。
Why because computers are better at numbers than trying to figure out a whole like path。
is much slower to do that。 And then there's the file layer which is where like the bits themselves what they consist。
of on the drive like your actual files and then there's the block layer which is like。
where on the disk they are。 All of those different layers come together to make a file system and you have to at least。
understand parts of those layers or at least how your layer if you're working on one particular。
layer interacts with the next layer down。 Hopefully not more than a couple layers down but that's you have to understand that when。
you're doing that。
Alright what else do we have on modularity and layering。
So this is one that we don't really go into this much detail maybe you probably do in。
a compiler class but I'm about to show you some layering that happens in g plus plus。
You know how you have your program and it turns it into a binary。
Well there's lots of steps in there。 It's not just like it just goes through once and turns everything straight into binary。
There's lots of different parts and we're going to see some of those parts in particular。
so that you can actually see what the compiler actually has to do。
First we're going to look at is the preprocessor。 So the preprocessor is one of the ones that all the pound includes or pound defines or。
as you yarkshows call hash includes or hash defines。
The preprocessors what takes those and manipulates them and changes those into something else。
that the next layer down can do。
Let me show you what I'm talking about。 What we're going to do is we're going to go over and we're going to do a little program。
called hello。cc and it's going to be very simple。 It's going to be like day one of one or six be here。
I-O stream using namespace STD。 And then let's do a pound define because we've got two pound things here。
They include and then the pound define。 I'll talk this K range start 100。
And then it's main in to our C in a char star RV。 And then let's do that for int i equals K range start。
I is less than K range start plus 10。 I plus plus。
And then in here we'll do see out hello world and then we'll actually do the I as well like。
that and that's it。 So pretty straightforward program。 Anybody see any bugs？
Okay。 What I'm going to do I'm actually not doing this on myth。
I'm actually doing this on my computer because I'm going to use a slightly different compiler。
I'm going to use a compiler called clang which allows you to break out some of the details。
here that G plus plus for some reason doesn't let you you might be able to I couldn't figure。
out the way to put in clang you'll see it。 So if we do the following clang dash E hello。cc。 Okay。
So let's do this first clang hello。cc - oh hello。 Make sure it works and if everything works right。
Oh no。 Maybe any of you maybe need to do clang plus plus。
There we go that works so I guess clang plus plus work。
So if we do hello in this case right it will do that and that that's makes sense。 Okay。
That's what I'm going to do。 If we do let's try clang this time clang dash E hello。cpp。
This is saying run the pre processor that's one of the deals with the pound and close and。
pound of funds。 Okay。 And why is this not working？
Oh， CC that's why there we go。 There we go。 Okay。 So let's go and buy。
Here's what it's actually doing。 Okay。 So what it's doing is it's taking a bunch of the well first of all it's taking the pound。
include and including it。 And guess what that pound include header file probably also has includes and it needs。
to include all those and it just basically creates one more file that it's all the stuff。
that you're about to process in one file。 Okay。 And it does that。
In fact let's see how big just the pre processor creates word count 41，000 lines。
Comes out of our one hello world program just from taking that pre processor in there。
But let's take a look at without。 Hello。cc。 Let's take a look at it without the see out in there and without the just just look at。
what happens to the K range start in here。
All right。 If we do the same thing again I don't 18 lines so that's much better。
Okay。 So what we did this stuff at the top here is just it's other information that's going。
to be used by other stages of the compiler so file amounts and so forth。
And notice what it did with our pound define。 It took that K range start and replaced it with the 100 and it replaced the 100 here。
and that's all it did。 Took whatever you define and just went through the entire program found where it was defined。
used and then replaced it。 It's all it does。 It doesn't do anything else in this case。
It might do it might actually do a little bit of replacing a variables if you if you set。
it up that way。 But that's it。 So that's the pre processor does and that's the first stage。
Okay。 The next stage is called the Lexer and the Lexer is what takes the translation of a stream。
of tokens。 Right。 It takes all those the program and turns it into individual tokens of what's happening。
in your program。
Okay。 Let me show you what I mean by that。 Okay。 If we do the let's do let's go back and make it the original one again。
So we'll do that and we'll put this back just so we can see the original again。
Okay。 And let's do this。 If you dash clang dash X clang dash dump tokens and hello。cc。
What it actually does is it actually puts the output for some reason standard error。
So I'm going to actually pipe or redirect standard error to standard out which you may。
or may not know that you can do it but you can't and this you should use the two angle。
bracket ampersand one。
There it is。 And then we're going to look at it and here's what comes out。 Okay。
So what is it done here？ It is taken a type death called type death in this case and there's a type death in there。
that it actually created。 And let's see。 There was a char 16t in there that it created and more type deaths。
You know what？ This is actually now they look at it。
This is just the this is all the included file in here。
So let's get rid of that again and you'll see what it looks like with just the part of。
the program that we have and you'll see how it actually。
Let's see。 There we go。 Let's try this again。
There we go。 Okay。 Here we go。 So here's what it did。 It had to find our inch。
Remember we said that for loop in there。 What it did was it said if I was an int here and it found main and this is for the int return。
value for main and then it found main in here and it says that's an identifier main and。
the program is going to be look the compiler can be looking for main。 Oh， guess what？
It found a left parentheses。 This is the main。 Remember it's like int main parentheses。 Well。
that's the parentheses there and it calls it a left parent and it's taking it down。
to that level where it found each individual token and does that。 There's another int in there。
There's an Rc variable。 There's a comma。 You probably never thought about oh。
we have to deal with all this。 If you're writing a programming language that comma means something。
That parenthesis means something。 Well， it's just another token that gets taken into the thing。
We have some left some brackets in here for charge to our RV。 Let's see what else we have。
Let's see if I can get some more there。
Let's see。 Can we find any of the ones we had here？ Here's our identifier I that we put in there。
It has that in there。 There's the plus plus。 It actually calls it plus plus。 It does it。
So this is part of this by the way is so that it can do much more translation than some。
other compilers。 It breaks it down to this level。 But it does need to tokenize it。
That's a layering sort of idea。
Then after we go through the Lexer which gets all those tokens， then it goes into a more。
interesting part。 So far this is like oh， that's just translation。 That's not too bad。
Then it goes into this one called a parser。 A parser is now looking at syntax。 And parser says oh。
let's break these tokens into syntactically valid constructs。
Let's see what we do that。
This one is also exclaiming again。 This one is AST dump like this。
AST dump like that。 And now it actually is creating a little tree。
We call this an abstract syntax tree that actually has all different parts in there。
So let's see if we can find something in here that we've done here。 Here we go。
So remember we have a four statement in there。 Well。
the four statement is made up of a compound statement。 A compound statement up here let's say。
And then you've got a four statement and then a declarative statement in there which is like。
the individual parts of the four loop。 Let's see we've got an int。
And there's our 100 for our integer that we had the constant in there。
We've got to have a bool because there's a less than calculation in there。
So there's a boolean value that's going on when we're doing that。
We've got another int in there for probably the next part of the four loop。
Let's see we've got the 10 in there for the plus。 We're going to have a plus in there which is our binary operator。
So all these things somebody to think about how does the C++ language turn into this abstract。
syntax tree。 And when you build computer languages you don't necessarily have to think about this。
on some level。 But there are tools that do a lot of this for you as well。
And then you can see there's the post fix again and plus plus and so forth。
So that's how you end up with the parsing part of it which is actually a very interesting。
part of computer programming language design。
Okay。 Let's see。 There are a couple more parts here。
There is the semantic analyzer。 Okay。 The semantic analyzer part is right here。
That's the part that takes all those syntactically valid constructs and says does this actually。
work in C++？ Like so far it's taken it and saying okay this maps this little part of the tree and。
then it goes oh now I'm going to see if this works with a type system and if you can， so。
you're going to have to eventually put out assembly code so it needs to do that。 Unfortunately。
Klein doesn't show you the semantic analyzer part so that's that。
But finally there's a code gen part which is what you may have or what you would have。
thought about in CS107 which hopefully isn't bringing back bad memories。
But if we do the same sort of thing now instead of AST dump now we just want to do， I think。
it's dash S actually。 So if we do， let's not do the X claim to our clang。
I'll do plus plus again plus plus。 Hello， we'll do dash S。 Hello。cc。
Let's see if there we go and then yeah there we go。
Hello。S。 Well here's all your assembly language that came out。
Right and that's the final part is generating a similar。
What's cool about clang is it actually tries to comment its code。 It's kind of neat。
You know it's nice for them to do this。 It talks about the interludes and so forth。
Another interesting thing about this。 So let's look up， let's do this。
Let me recompile this using the big， by the way， let's see how long is this。
This one is 923 bytes。 If we put back in the two parts here just actually do the printing。
Let's see， there we go and now it is how big。 It's now 49 kilobytes。
Right so it changes a little bit when you have to all of a sudden print something out。
Right there's a lot more going on here。 But let's look for the C out function for instance。
There's our C out function。 Now notice something interesting about it。
It's not a good color。 Notice something interesting about the C out function。
It's got all this garbage around it。 Like underscore in record ZNST3。
You didn't see that when you saw your functions in C code for an assembly or whatever。
The reason is because C++ allows you to overload functions。 Right？ So you can't have just one C out。
You can have 10 C outs that all have different parameters and things。
So what it needs to do is it needs to do what's called name mangling。
And it actually mangles the name and keeps track of it of course。
But it mangles the name so if you had two different or 10 different C outs it would know which。
one you call and you can't do that in C。 It's one of the reasons we do 107 in C instead of C++ because it can't do that。
It doesn't do this and it's much easier to read this sort of code。
So that's one thing you can look at in there。 So there you go。
That's the different parts of the compiler。
Okay？ So you have all these different parts that happen and it's all about layering and it。
does one thing at a time and does multiple passes across the code that you can then use。 Okay？
Then uses to actually build your assembly code。 It's kind of amazing to me that compilers go as fast as they do。
I mean your computer is already fast but making them go fast enough to compiler code。
just like that and when you think about all these different things that happen。
Oh and by the way that didn't even talk about compiling with -03 in there or some other one。
If we had taken out the C out statement again and did 03 you would have had like a 20 byte。
file because it would have optimized everything out because nothing actually got printed。
So all that optimization takes more time and so forth。
And if you take a compiler's class you'll learn this。 Compiling time is not really that important。
It's your runtime that becomes more important。 So it does take more time to compile something that's actually okay。
Now you do have to worry about programmer efficiency and you can't take days to compile。
things or hours like it used to but it doesn't matter if it's a little longer to compile as。
long as it optimizes to make your program actually faster。
If you only have to compile once you run it a million times。
So that's how that works。 Okay。 What else are we going to talk about？ Computer networks。
So that's a layers。 Right？ There's tons of layers in your computer networks here。 Okay？
There's the TCP/IP layer， right？ Which is the data of how packets get transmitted back and forth。
I talked about this the other day where it's the TCP layer that says， "Oh， here's your。
packets in a particular or in what order？ Like zero through a hundred packets。"。
And it's that layer that on the other side gets the packet and then says， "Oh， I've got， the packet。
I've got packet seven。 I'm going to send an acknowledgement for packet seven back to the original calling。
the， computer that sent me the packet。"， And so forth。 So that's what the TCP layer does。 Okay？
But there's all these other layers in here as well。
There's the application layer which is what you were you writing your application when。
using these sorts of things。 There's the transport layer which is where the TCP happens。
There's the network layer。 The network layer is the part where it says， "Okay。
I want to send my packet to a particular， computer。
but first it needs to go through the next closest router， like the one that's。
in the ceiling or whatever for your Wi-Fi packets and so forth。 Okay？ In fact。
let's take a look at how you might figure that out or how you might do a little。
analysis there。 There is a command which is Traceroot。 Did I show you Traceroot before？
I didn't even smart class。 Traceroot。 If you took 106B with me， you would have seen this。
Traceroot is a way to find out all the different paths that your or the different path， one。
particular path let's say that your packets might go through。 So I happen to find a engineering。
new university of New South Wales in Australia， edu。au。 If we try to trace the packets here， "Oh no。
it's not going to let me。"， This happened earlier when I tried this at my desk。
Sometimes it fails and it won't let you。 So I think， let's see， there we go。
I did do it earlier just in case this would happen。 So let's pretend that didn't fail。
You have to be prepared。 So what it did was it actually， if it worked。
it would have shown you all these different， routes here， which basically says， "My computer。
which is 10。34。160。2， needs to go to， or I guess， that might be the router actually。
My computer might first go to the router。"， And then it goes to something at S soonet。
which is still on Stanford。 Then it goes to another 171， which is still at Stanford。
Then it goes to another soonet， which is still at Stanford。
It takes four hops just to get off the Stanford campus if it's trying to go to Australia。 Why？
Because we have a lot of computers on here and a lot of different ways you need to go through。
a lot of steps just to get out of your local network。
Probably that is because we have not enough IP addresses to go around。
Then it goes to this thing called scenic， which is a， I believe that's something to do。
with the universities in the Bay Area and kind of the North， the Pacific， Northwest Pacific。
somewhere。 Then it has to get to Australia。 So it goes to this place called Pacific Wave。
which is a， which is a ISP or a router that， basically sends things across the ocean。
And you can actually see， if you go， let's see if I still have this up here， there we go。
You can actually see all the different submarine cables that are around。 You might ask， so wait。
why don't the， I always thought these like signals to go to Australia。
would go through like satellites and things， right？ Sometimes they do。
but it turns out it's much faster to go through the underwater， believe， it or not。 Any ideas why？
Actually， the radio waves are actually faster in the air than in underwater cables。 Any ideas why？
How far away are geosynchronous satellites？ Anybody have any idea？ How many？ 22，000 miles away。
It takes about an eighth of a second for light to go 22，000 miles up to the satellite。
And then another eighth of a second to go 22，000 miles back。 That's a quarter of a second delay。
And if you've ever called anybody， let's say in Australia and did get on a satellite connection。
there's a big delay there because it takes that much time to do it versus the 8，000 miles。
one direction to get from like here to Australia or 10，000 miles。 Much less time over the network。
But anyway， you can see， you can actually play around with this map and zoom in and zoom。
out and whatever and see all the different various connectors。
Air water cables are kind of cool in that they spool the model。 These giant spools。
they just have a boat driving along， spooling out the cable。 And every so often。
then they have to splice the cables together and run out of the spool。
And they do that for thousands of miles across the ocean and it just sinks to the bottom。
Sharks really like these cables too。 They can actually sense the electricity kind of going through them。
So they bite them a lot。 They have to break。 They break a lot。
So they have to deal with that too and they make them think of whatever。 Chase。
How do they retrieve the cables？ Yeah， when a shark bites it and like splits it。
they have a big hook that they know where， it is basically with GPS's over。
And they have a hook that they go along until they hook it and then they reel it up。 So yeah。
I don't know what it does to all the wildlife in the bottom。
But it's a yeah。 I think the different colors just mean。
I don't actually know what the different colors mean。
Good question。 It might be different。 It might be different companies that run them。
It might be a little bit。 Well， they are specific cables。
but like there's a bunch of gray ones here and I don't know， what the gray ones are。
I don't know what they are。 Yeah， the military。 Yeah， maybe that's it。 But anyway。
there are lots of cables around the world that because there are lots of people。
communicating and that's the way it goes。 But anyway， that's another layer。
You need to go through the network layer that sends these。
And this would then be the physical layer， by the way。
That's another layer that these things go through。 So another layer that we talked about。
I guess you don't put the layer。 There's the link layer， which basically is， let's see。
the link layer I think is between， the， I guess the network layers between the computer and the next router。
The link layer might be either internal to the router or not a guy at the show。
I'll have to look that on that。 But you can look up more different types of layers there。
We generally use the application layer， right？ When you do socket and bind and all those things。
that's generally the application layer that， we need to deal with。
Take a networking flash and go a couple more layers down。 Okay， 144。
What do you might want to do that？
Okay， what else？ The naming and name resolution。 Here's another one。
So we've already seen some of this about a file system， file system examples。 We've said， you know。
we've talked about absolute and relative path names and whether or not， humans like， Google。
com and computers like 74， et cetera。 We've done that。 URLs are also human readable。
Remember URLs can map to IP addresses with a file path name as well。 Okay， so there's that in there。
File descriptors。 A file descriptor is just a number， right？
But that number goes to a file descriptor table， which has all the other details and that's， that。
There's two different places where you can see naming and name resolution that you will。
see as you build more bigger programs。
Let's see， caching。 So here's a topic that I was actually a little surprised。
This topic wasn't covered in as much detail in 110。
I believe it's covered in more detail and correct me if I'm wrong， but people have taken， EEE 180。
Is anybody taking EEE 180？ Nobody has。 I think EEE 180 covers the caching in more detail。
But I wanted to spend 10 or so minutes talking about caching。
This is what I would have covered if I was designing this course from scratch。
I would probably cover this in a little more detail。 Here's how memory works on your computer。
You have main memory in your computer and it's a lot， right？
It's normally like 8 gigabytes or sometimes 16 or 32 gigabytes of main memory。
It's memory that is random access and goes away when you're computerized。 There's also， by the way。
the hard drive or SSD。 That doesn't go away。 It's super slow。 It's also much， much bigger。
A terabyte or many， many hundreds of gigabytes。 But there's main memory， which is the RAM。
Your computer also has these various caches。 So you might have a cache called an L2 cache。
You might have an L4 cache， an L3 cache， an L2 cache， and then an L1 cache， let's say。
And each one of these caches is smaller and smaller in amount of memory but also faster。
than the previous level。 What makes an L2 cache faster than main memory？ Number one。
it's generally built right on the chip itself， like on the processor chip， itself。
So it's close to end of wires or really close to each other and it doesn't need to go to。
some other bucks and so forth。 L1 is another one that's maybe built with also basically on the chip and also uses a。
different technology， maybe to make it even smaller or even faster but it takes up more。
space maybe or something like that。 And then finally you have the register file which you've done in 107 or 107E which is super。
super fast memory but you have 32 registers or something like that。
In fact there's more on different processors but some very small amount。
You might have something like， I don't know， 8 kilobytes or maybe 128 kilobytes of L1 and。
256 kilobytes of L2 etc。 What these are therefore is when you're requesting data。
you generally request data often， the， same data sometimes many times。
If you have a variable I that you're looping over， right？ If you're looping over 100。
000 times you're going to access that variable 100，000 times。
So you might want to keep it in a register or keep it in an L1 cache somewhere where it's。
going to be really fast to access because you don't want to have to go to main memory。
and go look for it。 Here's the difference between main memory and the registers。
There's a good analogy about this。 Main memory， let's do it this way。
Let's say that registers is like if I had this pen and let's say this is a register。
I have the pen sitting here and I want to write something。
Going to a register I have to pick up the pen and I have to write something and then I。
put the pen down。 Going to the L1 cache， it might be like the pen is way over here on the table。
In order to go to the L1 cache I have to go over and hit the pen and then come back and。
have to write it down。 That's how much slower the L1 is。 L2 might be the pen isn't on the table。
it's over in gates or something。 I have to go back to my office。
I have to go to the pen and I have to come back。 That's how much the floor is for L2。
Main memory would be like let's see the data is in San Francisco and I have to walk there。
to get the data。 That's how much slower is。 A million times slower like in some cases。
You have to go and walk down San Francisco and come back to the pen。 That's it。
Drive is like the data is in New York City and I have to walk to New York City and go， get it。
That's how much slower it is to get something off your hard drive or your solid state disk。
than it is from the register。 You think it's pretty important then to keep things as low in memory as possible if you're。
using that often。 What I wanted to do is I wanted to show you a little example of how a cache actually works。
Some of the numbers that might go into using a cache。
You do not need to know any of this for the final exam or whatever。
I just want to show it to you just to give you a feel for how caches actually what they。
have to do to keep the data。 What we're going to do is we're going to do a thing called a direct mapped cache。
Here's what this is。 Let's say that you had 32 kilobytes of memory。
In order to do 32 kilobytes of memory we would need to have five bits of information。
One bit per byte as it turns out or one bit for two to the fifth number of kilobytes or， whatever。
You need five bits of information there。 Like zero zero zero zero zero zero zero zero zero zero zero zero zero zero zero one all。
the way down to one one one one one。 Let's say that was your main memory。
What we're going to want to do is we're going to want to have a smaller amount of memory。
Sorry not 32 kilobytes 32 addresses that would make more sense so you can do five bits here。
32 addresses but let's say that we have a cache that only has space for eight of those。
memory like sections。 Let's do it like this。 Let's say that you had eight bits in here one two three four five six seven seven one two。
three four five six seven eight eight。 How many bits do I need to represent eight different locations？
How many？ Three。 You need three bits right。 We're going to have this thing called a tag or an index rather an index which is going。
to be zero zero zero zero zero one and I'm actually sorry I'm going to do this。 Oh no。
I'm going to do it。 Hang on。 We're going to we're going to have the I put it in the wrong place here。
You're going to have a the index here or maybe I took it in the right place。
Anyways the index here and the index is the part that tells us yes so zero zero zero zero。
zero zero one zero zero one one zero zero one zero one zero one zero one zero one zero， one one one。
There's our eight locations that we're going to be able to store some of those 32 addresses。
Zero is going to map to zero one is going to map to one two is going to map to zero one， zero etc。
Where do you think eight seven is going to go to one one one one where do you think eight。
is going to go back to zero right we need some modules here to actually do that thing。
we're basically going to store them based on the upper three bits of their address okay。
and so I'm sorry it's actually this is where I got confused it's the lower three bits of。
their address that we're going to do。 So basically zero zero zero is going to go in zero zero zero zero zero zero one is going。
to go there etc。 Okay and remember what we're trying to do is we're trying to say every time you ask for。
some memory it's going to first look in this very fast cache and if it finds it it's going。
to return it faster than if it had to look it up in main memory。
Okay at the beginning nothing is in your cache okay we are going to try to look up the following。
ten different values okay we are going to look up one zero one one zero then we're going。
to look up one one zero one zero and then one zero one one zero and then one one zero。
one zero and I got the rest of them here too we're going to look up one zero zero zero zero。
zero zero zero one one one two three four five six we need four more we're going to be one。
zero zero zero zero one zero zero one zero one zero zero zero zero zero what are you doing。
class day listen to Chris this talk much better and then one one zero one zero okay we're going。
to look up all those numbers look look up all those addresses and if you look it up and。
it's not in the cache it's called a cache miss and that's expensive we want to get cache。
hits which means that it's in the thing because this is direct address it actually turns out。
we can't we can't keep the values in there very long there's other kinds of caching which。
allows you to keep the data in there longer we're just going to see this example see how。
it works here's what you do you go to let's see if you look up one zero one one zero we。
look at the one one zero and we go over here and we find the index and this is called the。
tag by the way and we look up we say is one one zero fill or not well one one zero is not。
filled so what we do is we say first of all we say that's a cache miss and we put the。
one zero well we put the let's do this we put the memory for one zero in here with the one。
four one zero one one zero the memory for that in there and then we're going to tag it with。
the one zero to differentiate it from the other things that might fit in that bucket okay so。
we looked up one zero one one zero we looked at the one one zero we did it in the wrong。
one didn't I guys got to catch me on this we did it in there did in the wrong one one。
one one zero and we looked at one zero and then we're getting the memory from one zero one。
one zero okay that's what we're putting we're putting that memory there so when next time。
we look that up we will get it quickly so let's look at the next one one one zero one zero。
we look at zero one zero and we go to zero one zero is it there no so we it's another。
miss and we do the same thing here we put we've got one one and then we have the memory from。
that location doesn't matter what that is right now okay and then now it's in the cache。
let's look up one zero one one zero again we go to one one zero it's there is one zero。
the tag it's there we quickly return this value that is a cache hit and that's a good。
thing and that means that it's that it's we've got that value in there and you can think。
that if we're going to access that number many times well we'll always find it in the。
cache until it gets replaced by something else。 Let's look at the next one 11010 we。
look at the 010 0 0 10 1 1 it is also in there already so that's another hit。 Okay let's look。
at 100 0 0 0 not there so we put the 1 0 there and we have the memory from that location that's。
a miss 0 0 0 1 1 0 0 0 1 1 it's a miss and we'll get to one that's interesting in a second。
1 1 the memory there and then 1 0 0 0 0 okay 1 0 0 0 we've already seen we already seen。
that one too yes we did that's a hit okay and then 1 0 0 1 0 okay we go to 0 1 0 aha。
0 1 0 it's 1 1 but we're looking for 1 0 we now evict what's there in the cache what's。
they call cache eviction we send it out we put the 1 0 1 there we replace it with the。
memory from 1 0 0 1 0 okay and then let's do this one and that was a miss because it wasn't。
what we're looking for 1 0 0 0 0 0 0 0 0 0 it's a hit again and then the final one 1 1 0 1。
0 well we've looked at this one a whole bunch of times but now 1 1 0 1 0 we go back to 0 1。
0 we have to kick this one out again and that's another miss and now we put this one back。
in there this method is great because it's fast to look up it's not great if you happen。
to get two pieces of data that are both mapped to the same bucket at the same time anyway。
that's how you do this mapping a lot of times this is supported with hardware so these things。
can be done in parallel a lot of this can be done in parallel and it's also there are。
ways to do it so that the values can go in any position in the cache and you have to determine。
whether or not to kick something out of the cache based on how long it's been there if。
it's been there most recently then you don't kick it out it's been there a long time you。
kick it out or whatever and you have to go through that so I just wanted to show you that's。
a little of the math that you might have to go through if you take E。D。 180 and you work。
on how do caches actually do their thing okay so that's that one okay questions on caching。
yeah very good question the question was if you ask if you do malik where does it go。
to come from totally depends generally if you haven't if the if the computer hasn't used。
that memory location recently then it will have to go to main memory to request it if。
it has been used for maybe from other programs or whatever but it's not being used by other。
program now it might still be in the cache but if you do malik it might get a bunch out。
of the in fact it won't actually need to read any information out it says hey here's。
your locations first time you try to read from the location it will populate in the cache。
the next time it will be there hopefully it will be much faster and by the way those。
four levels of caches or whatever it keeps bumping it down down down down down as far。
the more you do so your data will be in all the upper level caches but looks in the lower。
spot first doesn't find it looks in the next one doesn't find it and then eventually gets。
up to where it finds it yeah good question any other questions on this okay let's look。
at so or let's see what board there we go okay so that was caching virtualization yeah。
yeah so the question is is your stack moved to your cache surely right I mean the cache。
is separate from your program itself right the caches based on how often you use that。
data and if you use it it gets looked for in the cache it is not there it gets put in。
the cache once it finds it and so again there's various strategies for doing this what you're。
trying to do is you're trying to get about if you can ninety percent cache hits in your。
lower level caches so that's very fast and this really does speed things up and significantly。
yeah yeah so yeah the question is are these being virtually mapped in here and they are。
they are all being virtually mapped in the addresses that are stored or the memory addresses。
that are stored in the caches are actually physical addresses so a particular program。
will have a physical address now another question that might come up is wait what happens when。
you do multi processing and you shift off the process do all the caches remain or do they。
go away generally to get cleared for a different process they might get saved and popularly。
populated but generally I think they just get cleared so every time you do a process switch。
your caches get cleared which slows things down significantly too they talk about you。
talk about caches as that being hot or cold a cold cache doesn't have any memory in it yet。
the hot cache has all the memory there it's really fast to access what other kinds of。
caches by the way we have you did a web browser cache or you did a web web cache web proxy。
cache DNS caches is whenever you're trying to request a Google's address trust me your。
local cache has Google's address because you use it all the time so it's a need to go find。
it and then there let's see oh there's another one called them cached which is I don't know。
much about that but it's for web content that keeps the web content cached in a slightly。
different way so there's lots of different places where caching becomes important and。
it does significantly speed things all right what else can we do virtualization so we I。
mentioned virtualization a few times you've seen it in labs and so forth as well virtualization。
there are two types one is an abstraction that makes many resources look like one okay let。
Let me give you an example。 There is a hard drive system called RAID， r-a-i-d。
And it basically says， let's say that you have a computer， and it has many hard drives。
Why would you have that？ A couple of reasons-- you might have four hard drives。
associated with one particular server。 One reason is that you can keep file one here。
part of file one here， and another part of file one here， and another part of file one here。
And maybe throughout those four-distits， duplicated in such a way that if this， entire disk dies。
then you can recoup file one just， with the other three disks。
It's very good for redundancy reasons。 But let's say that you--。
another reason you can do it is if you kept file one here， and file one here。
remember reading from a hard drive is slow。 What if you can do it in parallel and have one process。
reading from this drive and the other process reading， from this drive to get part of the same file？
You can actually do it almost twice as fast， because both disks will be running in parallel。
to giving your data。 So it's both for robustness and for safety， as far as robustness and for speed。
rather， so that you can get your data faster。 But by the way， your file system。
when you access a file， it has no idea that there's four disks here。
It looks like just when you make one request， and it gives you the data back。
Internally to the RAID system， it does all these special， oh， there's four disks。
and I have to do that。 That's a virtualization。 Four disks look like one disk， but they。
are partitioning such a way that it， is either faster or more robust。
So that's one type of virtualization。 The Andrew file system， this is what。
we use on the MIF machines。 If you ever noticed， if you've logged in a MIF。
or you log in to Cardinal or if you log in a MIF 52 or 58， or whatever， all the same file system。
the reason it's all the same file system， is that it's a file system that is spread across。
all those computers。 In fact， Andrew file system is spread across the world。
You can go to Carnegie Mellon's--， if you can do CD from your MIF to a machine in Carnegie Mellon。
now you don't have the permissions probably to do anything， but you can actually CD into it。
We always type slash user slash class。 Really， that's actually a link to slash AFS slash IR slash。
et cetera， et cetera。 And those are like AFS is the Andrew file system。
I think IR is Stanford's or whatever。 And so it's mapped to across the world。
But that's a virtualization。 It looks like there's one file system。
but really there's thousands and thousands， of computers connected to that one system。
They share the file system。 Web server load balancer。
This is when you log in a MIF and we did a load balancer in class。
We log in a MIF and it comes up with--， sends you to MIF 52 because it's got the fewest number of people。
on it that are working。 Virtualization is also making one resource look like many。
So what does that mean？ Well， this would be like virtual to physical memory mappings。
So you know how we've said last time， that you have two processes and they both think。
they have all the memory in the entire system？ Well， they don't。
They only have a little portion of memory， but it's making that one--。
it's making the one resource look like many resources， instead of the other way around。 Threads。
Well， it's thread has its own stack segment。 Well。
that's just the irregular stack broken into many pieces。 That's virtualization。 Virtual machines。
So who has used VMware before？
I think I have VMware on here。 Here we go。 Here's VMware。 I can load-- if I load this up。
I can load a Linux machine。
on my neck that's really Linux。 And it really thinks it's talking to hardware directly。 And in fact。
it is， because Intel sets up things， to make it virtualization work right。
But now I have literally two operating systems running。
on a computer at once-- my Mac operating system， and a Linux operating system。 And by the way。
VMware was--， the company has started from Professor Mendel Rosenblum。
who's here and teaches this class， occasionally in some other systems classes。
He did pretty well creating VMware。 But anyway， now I've got-- see， now I'm just inside my Linux。
machine up here。 And I also can do-- you can do windows or other machines too。
I don't even have a Windows PC， but I use Windows programs， occasionally。
And all I do is run it out of VMware and makes it easy。 So it's a little slower because there's。
some translation that has to happen。 But it's not too bad。 And it works pretty well。
Works pretty well。 That's how the ends work。
All right。 Concurrency。 We've talked a lot about concurrency in this class。 Multiple processes。
multiple threads， multiple cores on your multi-processor computer。 Wasn't until about 10。
12 years ago， you really couldn't buy a multi-processor computer。
for less than tens of thousands of dollars。 It was really expensive。 In 2006。
Intel started pushing out a dual core machine， which means now you've got a computer that can literally。
do two programs at the same time。 And that's concurrency。
Signal interrupt handlers are all about concurrency。 Signals， your program is running along。
with an interrupt handler。 The kernel is doing its own thing。
And then signals your handler to stop the rest of your program。
and handle whatever the child process handler， or whatever is happening there。
There are languages which are built specifically， to do concurrency。
There's a language called Erlang， which， is a great language but kind of hard。
to get your head wrapped around。 But it basically doesn't allow race conditions。
You can't make a race condition early， because of the way the languages built。
Not bad if you're trying to do lots of concurrency。 Yeah？ So signals interrupt， but it's not。
because they can try and code that works， before they throw a problem。 Good question。
If signals interrupt handlers， just like everything else。
If signals interrupt handlers are concurrent， how did it work before multi-processing？
When you have one processor， you can mimic multi-processing， by time slicing everything。 Right？
So the kernel gets a little time， and you get a little time。 The kernel gets some time。
and you get some time， in your single manner， get some time， whatever。 So that's how it works。
But anyway， Erlang， look it up。 It's an interesting language if you。
want to do very safe concurrent programs。 JavaScript is exactly the opposite。
They said concurrency is too hard。 We're just going to have one thread。
And it's not quite as true these days。 But it made it very easy to program in JavaScript in some ways。
but also not very efficient because it has one thread。 It does everything。
So that's that。 OK。 All right。 And then finally， we have client server response， which。
is a request and response。 This is what we've been doing for networked。
This is exactly what your map reduce is doing， where your map reduce main program is talking。
to the reducers and the mappers back and forth， until it gets all the data back。
It's a client server request。 You see this all the time， not just in networking。
You see it with multi-processing。 You see it with processors that have。
to basically talk between each other， and coordinate things that happens。
System calls are also another request and response， right？
Where you say a system call and the kernel， goes and handles it， you are basically。
talking to the kernel that talks back to you， that gives you， back the answer。
And there's that handshake that has， to happen via a system call in there。
All of the networking protocols， of course， are various examples of client requests and response。
And then the file systems that we just talked about。
are also client requests or a request and response。 Because you look for a file that might not。
be on the hard drive that the mitzv are using。 It's on the file system that the hard drive。
is up some other building， which might be--， it might be if you're looking at Carnegie Mellon's。
it's on the hard drive over at Carnegie Mellon。 You have to do some requesting and it's kind of slow。
in that sense。 So those are the seven pieces that we covered this quarter。 And there's a lot there。
but you know some of the details， about that。 You still don't know some of the abstraction underneath the hood。
sort of details。 You'll get those in other classes。 But in here。
hopefully you can see how many of the things， that we've done relate to these ones。
What questions do you have at this point？ All right。
So I will then see you for last class on Wednesday。 It is on non-blocking file。io。
And then we have the final exam。
No laps this week， there are no laps。 We'll see you then。
P21：Lecture 20 Non-blocking I_O - main - BV1ED4y1R7RJ
Okay， all right， so we've made it to the last day。 Woohoo！
I'll have more comments right at the end of class about what this class， where you should。
feel that you are after taking this class and so forth。 Without raising your hands。
unless you absolutely want to， how many people in here came in， to CS110 and what， man CS110。
I've heard that it's so much work and it's going to， be like， yeah， I see a couple people going。
I don't mind telling you it's going to be， not work， but it's been okay， right？
Like you've made it and you did very well overall， I think I'm very happy with the way。
students have done and how people have gone through the class。 As I think I might have said before。
it's not me speaking up here that's teaching you， all this stuff。 As much as I'd love to say， well。
I'm the one teaching you all this stuff， it's you doing。
the assignments really is where you get the learning out of this， right？
You don't learn about six-spend from me telling you what six-spend is， you learn about it。
from like， wait， how does this work and how do I program？
What's this weird race condition and why is my code stalling and why is there this deadlock。
and whatever， you learn that from actually doing it and putting in the hard work。 So good job there。
I'll have a couple more comments about that later。 So how's the assignment going。
MapReduce getting there？ Okay， some people are like， I haven't started it yet。
you should probably start it soon if， you haven't yet。 Technically it's due today。
I put something on Piazza that said， look， no， or somebody。
else answered this on Piazza that said that you won't have any late days until starting， Friday。
So you still have until tomorrow without any late credit for the assignment if you're still。
working on it。 We'll have office hours both today， right after class。
for about an hour and then tomorrow， morning my normal regular office hours。
I'll hold those as well。 You are welcome to stop by and chat about the assignment or about the final exam and。
so forth。 And the exam is Monday， so it's coming up pretty quickly。
There's not actually that much time before the exam but it's Monday。
If you do have extenuating circumstances and need to take the later exam and so forth。
please let me know before too long。 If you have other accommodations。
I will email you in the next day or two about timings and， so forth。 Okay， yes， question。
Are there going to be any office hours over the weekend？ Well， that's a good question。
The question was， are there going to be any office hours over the weekend？
We will definitely have something on Sunday。 I haven't talked to the CAs yet about what other plans they do。
We'll have some。 There's also going to be a review session that I think the CAs we're talking about Saturday。
at some point or something like that but we'll let you know as soon as we can about that。 Yeah。
so that's that。 And then let's see what else。 We will have everything graded midweek next week as best we can before that。
As far as one of their logistical thing for the end of term evaluations， please do that。
if you get the chance。 I will gladly take any constructive feedback and constructive feedback is like。
oh， you， know what， there was too much material or this material didn't make much sense when。
you covered it。 And I mean， I'm not upset about good constructive feedback。
It's feedback like the question is not going to help me much。 But like feedback like， hey。
I wish you Chris would have done this or this was good but， this could have been better。
That sort of stuff fine with me。 I'm always trying to make the class better。
So I'll appreciate if you did that。 Let's see。 Okay。
so today we're going to talk about a topic that's not on the final。 And in fact。
I was looking at it。 I was going over it this morning going。 It's way too much for one lecture。
There's actually too much stuff。 I'm going to skip some things in here and basically give you the kind of the overview。
we have a couple of examples of this non-blocking IO。 So so far we have been talking about IO。
In other words， reads and writes that when you send the， when you do the system call for。
read or write with a file descriptor， it actually blocks until it gets a response or。
and the response is some amount of data。 Now it's not always all the data you're either sending or requesting but it's some of the。
data or at least it may， it's going to be some data and it waits until that some data。
is actually processed， be it over the network or reading or writing a file。 And today。
and the way we get around those limitations is we have just been doing multi。
threading or multi processing which generally multi threading where we say， oh great， let's， run。
we're going to have this reader write but it's going to block。 So we're going to have。
we're going to have the ability for other threads to accept connections， or to be also doing work。
That's how we're getting around it。 It's not the only way。
There is this idea of having a non-blocking situation where you say I'm going to read。
some data and it might come back with zero data but it comes back instantly。 Here it does。
it comes back with some amount of data and we're going to see why that might。
be an interesting idea。 And of course you can't do it just on your own program。
there's going to be some other， things we need to have support from the kernel and from the operating system about。
Yeah。 Question。 Yeah， good question。 This just came up on Piazza。
There was a question on a recent final maybe last quarter actually that mentioned the term。
I/O bound and we can contrast that with CPU bound。 Okay。 And I made a comment in the Piazza post。
You should know what I/O bound means and maybe we haven't talked about it explicitly。
But here's what I/O bound actually means。 It means that your processor is not chugging away doing calculations。
It's just waiting for some input or output to happen。 And that's this idea of this weight。
this blocking sort of idea is it takes time for， data to be processed。
Now I've mentioned on Monday that when you're reading and writing from a hard drive it's。
like millions of times slower than reading and writing from memory。
And so there is some what we call latency that is involved with actually doing a read， or write。
So something that's I/O bound means your CPU is sitting there twiddling its thumbs going。
and I don't know what to do right now because I can't go on until I get some data so that's。
what it means to be I/O bound。 The U bound is kind of the exact opposite。
It means that your computer is chugging away doing something and it can't finish some function。
until it finishes all of the calculations it needs to do or something that is taking the。
processors time。 If something's I/O bound if you can use that processor。
which by the way is really fast， to do some other work let's do that。
And that's where we have multi threading and that's why having this question came up the。
other day too about oh what happened when we had one processor how did we get this idea。
of multi processing like multi threading how did that actually work。
It was because your processor is so fast that it can switch between processes faster than。
you can notice so that when you move your mouse around the screen guess what that's a。
different process taking care of the mouse movement but you don't you can't tell because。
it's it happens so fast and it does such a small amount of work that it looks like it's。
just happening at the exact same time as something else。
And on a multi process or computer like the ones we have these days it probably is but。
on a single processor it was just that your mouse would move around and part of the time。
that it was would have been working on your program is now spent actually moving the mouse。
and updating the screen and so forth。 And this goes back all the way back to the mid 1980s when you bought the original of the。
original Macintosh computer it had mouse routines that it would look just like you know something。
would be happening in the mouse would take over for a second and you couldn't tell that。
it was time from the processor to take the time。 Yeah。
I mean saying something is I/O blocked is similar yes I mean the term I/O bound is well known。
as oh you've got a processor that's doing some input output and it's bound by the fact。
that that's low part is the input output that's really what I/O bound generally means。
And if you can if you can make something CPU pound bound that's not not necessarily a。
bad thing it's not neither one of these is a bad thing but being CPU bound is actually。
harder because it's much because your computer is so fast at doing things right now if you。
were trying to factor large numbers that that's going to be a CPU bound sort of thing because。
you take a lot of processing to do that right and you might be that you have to factor some。
large number and then spit it out to the processor。
So in fact I guess the farm program that we did with multi processing had elements of it。
that were CPU bound on the Python side right because it had to do all that factoring and。
then it never got back to you with a response until after the CPU part was done。
So so you can do that but knowing the difference between these two terms is going to be helpful。
long term anyway。 So yeah。 Alright， Kathy and then we'll go back here。
What if a program always be one of them？ Would a program always be one of them？
A program could at certain times be CPU bound and at certain times be I/O bound but it's。
going to be it's going to be one of the other generally。
It's all well it's generally going to fit into one of those although you could if you。
let's say you had you knew you were going to have a whole bunch of I/O bound threads。
just keep doing more threads until it becomes CPU bound or the trade off is not you know。
then that's perfect right you've got all these outstanding connections and you're you're。
chugging away creating all your processing all these outstanding connections and you've。
got so many that it actually is taking your process of time until you get something back。
So yeah I mean the trade off would be or the idea would be yeah keep doing see more CPU。
stuff while you can you know so now sometimes you don't want to do that because it takes。
more energy and there's other you know things going on your fans or come out on your computer。
and whatever but if you really wanted to eke out every processor cycle from your computer。
you would keep doing more threads to be able to do that。
I mean I could ask you you know I could ask you I could say why is networking going to。
be I/O bound and you would probably you would have to say oh it's because it's the slower。
part of the connection it's got to talk between two computers and it's not you and your processor。
has to wait and we use six to spend and we use you know we use other blocking sort of。
reads and writes and so forth so yeah。
Other questions on that good questions。 Okay so let's talk about I/O in general and system calls。
We have this idea that we have fast system calls okay those are the ones that return。
immediately which means all they need to do is do some process that doesn't really involve。
a connection or file reading or writing and so forth。
So there are lots of examples of system calls that might do that but let's say get host。
name or something like that that system call doesn't need to go outside the network it needs。
to just query something that the that the kernel already knows about it just returns。
immediately and that's the way it goes。 There's no real limit on immediately as long as it needs to do stuff as long as the processor。
needs to be doing things or the kernel needs to be actually doing something to do that that。
still can be considered fast even if it takes 60 seconds it still might be fast because it。
has to do something so you're factoring a whole bunch of numbers or you're factoring。
one number it might be still fast in the sense that it's not waiting around but the idea。
of a slow system call is when it actually has to say I can't do anything right now until。
something else responds to me or you know gets back with some data and so forth okay。
So slow system calls are the ones where we don't know how long they're going to wait。
it depends on some other resource right reading from a client might depend on some other computer。
getting back to you and the and we want to minimize that kind of behavior if we can help， it okay。
So calls to read are fast if they're from a local file because it's not that much but。
you still depends on what your definition is you can still consider that slow because。
file reading is slower than memory access and so forth but in general it's relatively fast。
What calls are slow if it's going to a socket or if there's some internal congestion like。
the socket is filled with lots of in going and outgoing connections and that can happen。
when you have one port responding to all these connections like a web server often has its。
port like it's really on the edge of oh my gosh I'm almost gonna not have enough time。
to respond to these because just the port itself is getting lots and lots of requests。
This is what happens when you go online to like Reddit and somebody puts a webpage and。
they're not ready for all the connections and it like suddenly goes down it's because。
well I just couldn't handle all the connections so we want to avoid that if we can。
And then slow system calls in general can block indefinitely in other words you have。
to wait for them and in that case they're considered slow。
We've done some of this before where we've made it so that slow system calls don't matter。
wait PID is a good one where we have that W know hang。
Remember before when we said oh if we don't if we do wait PID without W know hang and the。
negative one or not even the one just the W know hang it will wait until the child ends。
and that's a slow system call because it's waiting for the child end and it just goes。
on just gonna be off the processor until the child finishes。
So once we start using W know hang you have some logic you have to use but it means that。
you can get control back and just do it call it again at some other time after you've done。
some other work and then it won't and then you get the benefit of not having to hold。
up until that ends。 So let's see reading right with iOSocstream we've generally done it with multi threading。
to say great do the accept call but do it on in another thread so that you can go back。
and do another accept immediately。 Now we're gonna do it in a slightly different way okay accept and read slow the way we've。
been doing。 So we'll see we're gonna have to work up to this by the way about how this how this works。
So how do we actually make these slow system calls fast well we can tell the we can tell。
the system don't wait on an accept call what does that mean it means you call accept and。
then it returns immediately and you're responsible for doing it again if you need to actually。
get in a connection we'll see how this works it's gonna be a lot of wild true loops with。
some extra help from the operating system okay as we'll see and we've done some of this。
before in other forms so it will look very similar in a big picture。
So generally if you do accept if you set it to non blocking it can return with negative。
one which generally means an error and often it sets remember it sets error no if you get。
an error in this case it sets it to E would block which basically says okay it's the error。
is that it would block except there's no connection to block on so that's the error so。
it's not really an error but that's how you handle it we've seen that before as well。
Same thing with read and write you can set those to be non blocking but then it becomes。
your responsibility to call them again when you want to send more data okay now you already。
have to do that because all the data might not get sent from your buffer anyway but in。
this case it could be zero bytes get sent just because you it's not even ready to take。
your request at all so it just returns immediately and at some time in the future it will send。
the data okay so if it does if there's no more if it would block and there is data to。
be sent still you'll get this negative one business back okay if there's no more data。
you'll still just get zero as always okay so that's how it's gonna work let's see a couple。
examples of this let's do talk about this little program called slow alphabet server I'm。
not gonna type it up because of the time but here's what it's going to do we're going to。
artificially put some delay in here of a 0。1 second delay and a tenth of a second delay which。
is noticeable you can actually notice that and what we're gonna do is we are basically。
going to set up a client here to we're gonna have a thread pool like we've done before right。
and when we get a connection request all we're gonna do is set up a little output sock buffer。
and then feed the alphabet one letter at a time with a delay in there after each letter。
okay so what that's gonna mean it's gonna be 26 it's gonna take 26 letters it's gonna， take 2。
6 seconds to actually push out which is really slow to push 26 letters out but we're。
artificially doing this to see why it might be a good idea to use this non-vaught index。
okay so here's what it does it waits for a connection it has a thread pool to actually。
handle the connection so it can request it can respond to many connections the worker is。
slow because we put that artificial delay in there and then it closes the connection nothing。
here is actually non-blocking but it's definitely slow because we put that delay in there okay。
you will see this often servers in general if they're trying to serve many many web pages。
will give a little bit of data to you and a little bit of data the next one a little bit。
of data the next one so that it doesn't block one for too too long okay and you'll notice。
this sometimes if you're loading a web page it loads partial data and it loads a little。
bit of the image you're trying to download or it loads you know this this add and then。
this add and whatever it takes a little bit of time and that's you know what happened。
so you've seen this before okay you've seen these you've also seen buffering like on YouTube。
which is more or less the same thing where there's just not enough data that's gotten。
to you yet to actually give you an image or give you the the video so that's what happens。
there okay so here's a client that we're gonna actually test okay and the client here has。
a while loop and basically it goes and it tries to connect to the server which is going。
to be on local host here the same machine here and then it is going to read one byte。
at a time and then print out that byte but we have yet to do this non-blocking thing so。
the read it's going to read it pretty fast print it out pretty fast and then go back。
here and read it again the delay though it's going to come from the server doing really。
slow pushing out of the data okay and that's all it's doing it's printing out this out。
so let's actually see this if we let's see there we go okay so we have what was the name。
of this file this one was slow alphabet if we do slow alphabet server like this in the。
background and then we do see this one was this one was blocking alphabet client okay。
watch what happens there's your amount of like reading the 26 bytes you could tell it。
was slow it's actually time it okay if you time it then it will take in this case 2。6。
seconds which makes sense we delayed for 0。1 second for letter and it's 26 letters and。
so forth okay that shouldn't be too strange notice something in here we did put a little。
we counted how many times we called the read right we called it 26 times mainly because。
we asked for one character at a time okay is there a question over here oh good question。
yeah good question so I used a little bash shortcut if you do time exclamation next or。
do something with two exclamation it does the previous command after like it does the。
previous command we've time for instance so it just does the last thing this would probably。
do it twice time time yeah it does the last thing with it with the current command good。
question okay so anyway that's what's happening with the server okay nothing is new yet okay。
we basically done exactly what we've done before now I could do this up to 128 times before anyone。
would actually not get 0。2 seconds because I wouldn't even I'd be able to keep doing this。
as many times as we wanted to in fact I'm gonna cancel the server for right now but that's。
so that's where we kind of are already nothing should be new yet everything's blocking in。
the kind of the way it is we put this strange delay in there to kind of show that it takes。
some time and and it could be a network connection that takes some time okay question oh good question。
why did they use the why look I kind of wanted to show you that you have to do it one if you。
want to do it one bite at a time if you wanted to do a stream you could but that's all hidden。
from you and in this case and by the way it would ask for a big buffer is worth so it。
would still wait 26 or probably 2。6 seconds but not it probably wouldn't you wouldn't be able。
to see each letter it would probably ask for letter and then go okay I asked for the whole。
26 at once so in this case you want to go look let's just see it one character at a time。
all right so this is what we just ran and we saw that took 2。6 seconds and in our case。
we're only reading one character at a time so you can see each character and you can kind。
of perceive that it's going to be relatively slow okay so if we want to change to a non-blocking。
equivalent of this we're going to use a function that we are going to see in a few minutes and。
it's pretty low level but it's just called set as non-blocking so after you set up the。
client okay you get you call this function that place some magic system calls itself。
and says if you when you're reading from this socket don't block on it that's all it's doing。
we'll see that in a little bit okay but you're going to do basically the same thing here when。
you read though okay it is going to not block so you could get a negative one back okay meaning。
that it would there was some error but the error could be that it would just block okay so you。
could do that if you do get zero back it means that there you're still getting no or you don't。
have any data left so that's fine they're just like normal but in this case we want to if we had。
greater than zero we would handle it and it should be just one byte because we're still only。
asking for one byte and that's it but the other option could be what if it's negative one so it's。
not zero it's not zero it's not greater than zero so it's negative one check to see that it's not。
the oh it would block because that means there's some other error the connection died or there's。
some other error but if it is E would block then you just go all right in this case we're going to。
log that we had an unsuccessful read and we just go and try again okay this is going to do boom。
boom boom and try to read very fast yeah again is a sim I think he again and he would block or。
actually the same thing as it turns out I'm not sure they could be different on different systems but。
here's how you would figure that out by the way you would say air no - L and that lists them all。
let's look at E would block you would block rep E would there it is it's number 11 and in this case。
E again also 11 so it's actually redundant in this case although on different systems it might not。
be but either one of those could get could be the error number if it's going to block E again means。
try again basically in this case it happens to be the same number so I guess it's redundant on our。
system good question yeah good question all right so question this is the client right this is now。
the this is now the client they're doing so the client you know is setting it unblock now we we。
don't we could set the server to also be non-blocking and we'll see a little bit of that later but we。
don't care right now the server is blocking because it's still fast we've done it with a thread and。
whatever but yeah good question yeah yeah we have to get there so this is good question good question。
wait is there is there any advantage to this we'll see an example later unfortunately because of the。
time I have to skip over some of the part that you can go look at the details but we'll see why this。
why we could use that we could see we'll see why this is just another way to do a similar thing。
we've already done is it better or not threads take a little time to spin up and threads take you。
know multi-threading things letting the operant letting the operating system handle the waiting is。
sometimes a better like the not the waiting but the the alerting us that there's data and so forth。
sometimes it's just a little more efficient so new servers generally handle it the way we're。
going to talk about just because they've decided that it's a little faster than doing that multi-threading。
so there's just this is really just another way of doing similar things we've already done the way。
we've already done it generally fine but if you're going to serve billions of web pages maybe you。
want to get a little more efficient this might be better way the way we're going to show you a little。
more efficient so long answer for why are we doing it it's another way to do it why is it important。
okay so anyway now if we run this this is the non-blocking equivalent okay if we run this and we do the。
let's see we still need to set up the server there's the server and if we call non-blocking alphabet。
client okay it looks exactly the same okay but take a look at what just happened we now got 26。
bytes back but we did what is that let's see 16 million reads in that 2。6 seconds right well。
why is that not why is that a good or a bad thing during the time when we weren't doing those reads。
we could have been doing other things so let's say our program needed to do something else。
right we could have done that something else and then gone back and read again later so。
otherwise we would have had to block right this is a dumb way to do it because we're just chugging。
away and doing all we're doing is reads but you'll argue that maybe you want to accept input from。
the user or something during that time instead of just blocking so the idea that this frees up time。
for thread city here this frees up time on the main thread in this case right now could you。
have a thread could you have called that read in a thread sure so it seems a little odd to do that。
but you might do that and then this is same sort of thing you might need a you might need to figure。
out when that thread ends and so forth in this case we're just saying you know what we're trying。
to read 26 bytes oh it's going to take a while maybe we want to do something else in the meantime。
right i mean what other things could we do non-blocking alphabet clients not that one how about non-blocking。
alphabet client dot cc right if we do the read this is the non-blocking read okay let's say that。
let's say that we get to here where it was unsuccessful maybe we did something else like。
i'm doing other work right like that and whoops there we go make and let's see i don't think i。
killed the server yet so that's that there we go it's doing other work doing other work right。
it's you'll see there are letters in there i don't know if you saw but there were some letters。
some fingers perched in there but look at how much work you did you know so we could you could have。
your one thread doing more work and this is the way to do it so that's the benefit you don't。
know threads necessary in here no thread pool no threads necessary we got the work done we did。
have to re-ask a lot of times but we were able to do more work that's probably the bottom one。
yeah other questions on that okay so where does that get us now well now we've seen that we can。
actually do this okay and we see that we get millions and millions of reads and we got lots of extra。
time because point one seconds to a computer is super duper fast okay how many is anybody in here。
read the book super intelligence by nick bostrom one of my favorite books although it scares me to。
death it's a book about uh one what happens when we end up creating a super intelligent。
ai it's actually smarter than humans and uh there's lots of ramifications of that and we have to。
maybe be a little careful about that if we want humans to actually persist but uh one of the。
interesting things about that they mentioned this that computers are so fast that you know if you're。
holding and i think this is the example they use in the book if you're holding a glass of water and。
you drop it right well because your neurons are relatively slow and the processing of your you're。
the seeing things is slow you don't get that much time to like move your hand down here and pick it up。
because you only see so many different data points now the nice thing about your brain is it's very。
good it's very good at integrating so if i throw a ball to you your brain actually can see the path。
and put your hand in the right place and you're very good at that two billion years of evolution has。
you know done a good job of that right but a computer your brain sees you know a hundred or。
thousand different data points in the point one seconds that it takes for somebody to drop a computer。
will see you know billions or millions of data points and so really to a computer dropping a。
glass is like the glass is like do do do do do do and the computer's doing like oh i can do i can do。
tons of calculations while doing that and i can you know it's so it's uh i mean that's the scary part。
is that computers are just super duper fast right that that's things but you know but they can't love。
not yet anyway not yet um so anyway read the read the book if you want like it's a little bit of a。
it's a little bit of a uh uh scary book to read but but read it it's good because it talks about some。
of things and especially if you're doing ai it's a just a great book to understand oh we might have。
to face this some on the future but my point is during the time when this is reading waiting for。
that slow thing to happen the computer gets to do zillions of other things which is the point here。
okay we could handle that in a thread or we could do something but now it's just one thread doing all。
the work okay now here's the part that i'm actually going to skip over a little bit um there is this。
class that we are going to that is used in the other example that basically is uh going to be used。
to show you how you can take do file reading or writing in this case uh piecemeal okay and so it's。
basically a it's just a little uh class that has an initialization that takes a uh a file descriptor。
and it takes the uh it takes the actual like data itself in a string format and it's going to push。
that data to that file descriptor piecemeal like some amount of data at a time it's going to do that。
and it's going to do that through this function called send more data and if the send more data。
however comes back with false it means you're done sending data but the whole reason for setting。
this up and and it's like many slides here showing you the nuances of this uh this outbound file class。
but the big the big idea is set it up so that you say i need to send this string to this file and send。
some and then it just continually can say send more data and it will do that kind of on its own and。
the whole point of this is doing that so there are lots of slides here where it talks about uh the。
details here here would be a little test program for that which basically um sets up an outbound file。
initializes it with a uh sorry it's actually a file name that gets you ready and not not the actual。
string but it's the file name it opens that file name up and then in this case it's going to print。
it out to standard out whenever you call uh send more data and in this case it's going to do that now。
it does this in a non-blocking way okay so it will send some data and then it will send some more。
and send some more but it will try to return immediately won't try to send all of it it might send one。
byte at a time or it might send a hundred bytes at a time but it's some amount of data so this is。
just kind of abstracting that away so that we can use it to show what happens when we do this with。
a server so i'm going to skip over all the all the details about this um the the basic idea or the。
example here takes a file and shoots it out to standard out so i can actually show you what the。
what the uh the end part of it is here if we let's see this one is uh it is outbound file test right。
and it just prints it out to the file but it did it in piecemeal and you can't tell because there's no。
delays in this one it just looks like cat but in this case it does it you just send one bit of it one。
piece of the data at a time that's all that's about no the details of this you can go look at it i don't。
think it's particularly important for getting through what we want to cover for the rest of the day here。
so lots of this um again you have to deal with accept and so forth as part of this because you're。
trying to send to uh that outbound uh server and uh let's see we're gonna just skip past this skip。
past this here's the by the way here's the set as non-blocking function okay i said that it was uh i。
said that it was low level it's another one of these system calls that's got lots of stuff in it that。
you if you want to actually use it then you go do it but the important part is here is that you。
call this function called f c n t l it takes in the descriptor and some other information and you。
you give it a parameter that says hey don't block on that descriptor when you're reading or writing。
from it so that's all that's doing uh there is set as blocking which does the exact opposite。
and then you can check to see if something is blocking or unblock or non-blocking as well。
low level functions don't need to worry about those details unless you care about creating them。
you know i just use that okay uh so again we're not going to worry about the outbound file。
distraction the idea is read a file every time you say send more data send a little bit of chunk of。
data that's all i care that you really know about this is all blasting through that because of the。
time but what i want to spend a little bit of time on is uh two servers that are going to utilize this。
outbound file uh class to actually enable you to serve data quickly without threads okay so the。
same idea as we did before or the same idea now is oh no more threads necessary let's do it okay。
we're going to use non-blocking IO for a single threaded web server again not necessarily the。
best way to do it but probably a little more efficient than using threads because we can do that okay。
it's going to be responsive because there's nothing blocking here and it's also hopefully not going。
to miss any connections okay that's because again it's got it's going to respond quickly but。
then you're going to check many times to be able to check and make sure that you you don't have a。
connection yet and when you do you get it and you handle it very quickly okay so let's see some of。
the details here of this uh function um we are going to create a list of outbound files in other words。
what this server is going to do is you have a web page and this is actually one that we can uh。
you can use from a web browser i'll show show it to you in action in a few minutes but you're。
gonna have a web page that basically is keeping a list of the files that you want to send out。
so think of it like a web server saying i'm gonna go to google。com or i'm gonna go to， www。stanford。
com and request a bunch of web pages each web page that gets sent back is going to be。
held in one of these outbound files that can get sent back piece by piece by piece okay and we're。
gonna keep a list of those uh in this uh in this program okay so what's happening here we are。
setting up a circuit okay or a socket rather we are uh then making it non-blocking now this is the。
this is now the server is now doing the non-blocking part okay it's going to set up this list of files。
that we're going to the connect the connection files the ones we want to send back it turns out。
they're all the same file in this case but we just because we're always just gonna request the same。
file we didn't say do a certain file specific file um remember in lab a couple weeks ago you did a。
file server you remember that example hopefully you covered in lab or you've looked it over this is。
the same sort of thing except now the file is all just one file basically the same same basic idea。
okay and then we've got a little bit of either logging or some other data capture thing okay。
there's a wild true loop in here okay the wild true loop will do an accept which will not block。
okay and it will calculate or just do the kind of the logging again of how many times it had to go。
through this wild loop okay if you don't have a connection it basically just goes it well if you。
don't have a connection what we want to do is use the time now to send some data to all those files。
so basically we're gonna go hey is anybody want to talk to me if not i'm gonna send a bunch of data。
to the files that have already been set up that's all that's going on here okay so we set up the uh。
we we send them out and in here we are going to push onto the list of if we do have a connection。
here okay then we are going to uh sorry if we don't have a connection here we're going to。
initial this one that looks like we're actually doing we are doing it when we do have that we're。
going to just get a connection set it up push it on your data and deal with it if we don't this is。
where we are going to send more data okay so basically uh you go and after you've tried to do a connection。
if you set the connection up great you don't set the connection up then you push some more data to。
all your files and just a little bit at a time and that's all that we're doing yeah um do you have a。
yeah i'll go back side what is the background between these things except for this great client。
this is a server remember this is the server going on here right so in this case we're accepting a。
somebody's requesting us so we have to do the accept this is basically where in every other。
program we've ever written we've said accept and it's just stayed right there until it gets。
until it gets a connection right now it's going oh great i have more time to do something because。
nobody wants to talk to me and i'm gonna do what am i gonna do if i get a connection i will set it up。
right if i don't get a connection then or if i even if i do the next thing i'm going to do it i'm going。
to take some time to push a bit to push out to all those files they're requesting this data and this。
is where it's going to now is this going to take some time yes now if you have a thousand files here。
that are being served it is going to take some time now by the way we're getting into CPU bound things。
if your list has uh 10 or 100 000 different connections now it's got to go whipping through this。
while loop a hundred thousand times that's going to be CPU bound because none of those calls are。
blocking it's just i've got all this stuff to do i have to do it in this case so this is this is。
where it would become CPU bound that's that's what's going on there okay all right let's actually run。
this if we run this uh let's do it this way if we run this but this one is called oh uh let's see。
hang on four i want to cancel the server before um this one's called the expensive server okay。
let me just run it and it's saying static file server listening on port 12345 and this is mid 61。
by the way i did restart all the myths today i had to go and like literally start half of them and。
they were all just sitting there dead i don't know why 107 probably not because of you guys um let's do。
this let us go to myth uh what did i say it was myth 61 and it is port 12345 and hopefully there。
there is the file so we are serving just this one file a million times and it happens to be that。
so if we go there and serve this file we'll do that now what's happening in here and by the way that。
was where it was it was doing that if other people also wanted to connect that might go up if two。
people are connecting at the same time but it's going to go through pretty fast okay what's happening。
right now this is myth 61 let's check something uh let's see ssh myth let's see see grab myth 61。
yes whoops yes and htop we should there we go take a look at what expensive server is doing right。
there that 100% is my expensive server why why is my expensive server expensive。
okay nobody's really requesting much data and what's it doing it is sitting here。
oops let's go back one slide there we go it's sitting here in this while loop not blocking on。
accept so it's doing this while loop again and again and again and there's no data to serve so this。
while loop is just going nuts okay if we put a C out in there we can see a whole bunch of C out。
statements just like we did before but what it means is that it's that if we do eight stop we are。
getting oh there it is 100% of the work is on one core and some myth machine myth 60 whatever this is。
is right now the fans are starting to come on and whatever and some poor 107 student doesn't get their。
their heat balicator down or whatever um so uh so that's what that's what's happening there so。
what do we want to do we've got this feature what did we do in the past when we wanted when we had。
to deal with this what did we do when we want to wait for something hell we moved it off the process。
right so we have to have this is the same thing there we have to have another operating system。
support for this okay and this is a thing called E-pol and the E-pol family of system calls are。
basically waiting for file descriptors okay so it's basically saying instead of doing that while。
loop again and again and again and instead of blocking on it we are going to be able to。
we're basically going to go and call the function we want when something happens so in other words we。
can we're going to do some sort of waiting if we have no data to send and we have nothing else。
going on we are going to shut the processor down or at least our process down and we're going to do。
that okay so there's a whole bunch of functions here there's a create function that returns a file。
the scriptor and it has this e-colo exact thing which we've used before and it sets up a。
file descriptor that we won't that won't that we will get alerted on as it turns out it has this。
watching facility which basically says what's happening whenever something changes so if you're。
watching a network connection if data comes in it will get the waiting will stop okay and you actually。
can call e-pol wait to do the waiting around okay and it's waiting until there's some time out。
which the time out you can set so that you can go do more work and whatever okay so it's basically。
the same sort of idea you've just now have more control over it than you would have from a blocking。
sort of server okay there's a struct in here there's this interesting see thing called a union。
anybody know what a union is you probably never used unions in c before it's a pretty old concept。
and it goes back to when data was scarce like when you had the data was scarce here's what this is by。
the way a union says this data structure can hold either something avoid star or an int or a u n 32。
t or a u n 64 t and whichever one you happen to have in there it will be able to get out that data。
based on which which one of these which one of these variables that you use in there so why does it do。
that so that you can store different types of data in the same amount of space right it's going to。
allocate enough for the entire the biggest thing even though you might only be using a four byte。
int right you can have it all have enough space for a 64 byte int but it won't but it but it doesn't。
need to have a different one for each one so there's an F4 in there it's just got one so look up。
unions and why they're used you don't see them too often but when you do you have to kind。
understand what they what they do okay we'll hold a single type of data out of the data types okay。
what kind of events can we listen for okay and this by the way goes but this is now event based。
things and this is we've seen events before when we have when we have signal handlers though that's。
an event based model but in this case we can say for a file the file is available for reading or。
it's available for writing and we can also have it what's called edge triggered and if you saw。
say if you took CS 107E you would have heard of edge triggered before but that means that you。
actually get like an event on a change in the descriptor okay versus what like what type of data。
you're looking for this is saying hey I want to do it when I get the data okay so what are we going。
to do we are going to set this up now where we have the the server itself main is not really going to。
change okay for this server it's going to set up the the socket and with create server。
server socket in this case and then it is going to run this server so this is just kind of abstracting。
in a way decomposing it a little bit okay run server is going to set things as non-blocking。
okay and then it sets this watch set of what we're blocking it's going to build the watch set which。
I will see in a second based on that server socket that we have okay what's happening with the build。
initial watch set we are setting up some E pole in other words to watch for that thing changing。
okay and then there's some other it's basically setting up the the union and the struct there in。
this case the struct and then it's calling this E pole control to set it up think of it in a very。
similar way to six suspend how you have to do the six the the set just set up the signals that you。
want to block and so forth same idea different topic but basically same exact idea and again。
I'm not asking to memorize this stuff right now or know how to use it but just know that it's possible。
to do this okay all right the next thing we're going to do is we are actually going to still do。
our wild true we're going to have these events we are going to have the number of events get。
triggered by this weight thing here okay we might have a timeout on there to say just wait until。
there's a timeout so that we can go do data as well you don't want to time it out but it could。
timeout and if it does well it'll just go on and do the next next part okay you can actually get。
many events triggered at exactly the same time think of it it's the same thing is the child。
process getting a weight PID where it returns multiple things so you should have you should。
handle all the events here okay and when you get them triggered we handle the events one at a time。
using a for loop so basically the events get triggered we find those events and then we go and。
and deal with that okay and if you get an event then you can actually in this case like consume。
the data read from the file description you say hey what did you want to tell me right and then。
it will do that and then you can publish the response if you don't have things to read in if。
you're just trying to write it out okay so what else is this and there's a lot of code here I know。
this is a lot of code that you have to do but you have to see but basically we are setting up the。
except there's a new except call by the way called except for which has this non-block。
parameter thing in there yeah mark of course oh sorry go ahead。
yeah well you'll see in a second why this is still non-block it's not waiting in the same sense it's。
basically you're you're doing it so that it's allowing how should I put it it's it's not it's。
if you it's not going and blocking on the except call you're still doing that it's just not going。
to have to call as many it's not going to go through this loop as many times you'll see when we get a。
little bit farther along good good question on this okay we again we do this is all set up and。
setting up the watch list and and setting up the the acceptance so forth okay so。
the consumer available data part is where it's going to basically do the get the connection。
okay and we're going to set the connection up and we're going to set the connection up。
at that point okay and it is going to and then we're also going to be able to do that's just setting。
up the connection and getting the request and then we are going to let's see do the。
where's here we're going to yeah still take care of the requests okay and we may have a whole。
bunch of active connections okay that we have to do and we kept that in a map and and so forth。
okay and then we will actually change the event based on whether or not we'd already have connections。
again these details go look at the slides I don't want to go into all the details about the。
how this this works under the hood but we basically are going to get requests or get the events happening。
and the kernel is going to handle this for us okay so where do we get here published response this。
is where we're going to actually in this case we're going to take do an actual website that we're。
going to respond with instead of the instead of that file in this case and the publishing the response。
is going to for all the different places that we need to send data send a little bit of data at a time。
using the right command okay it's only going to send let's see I believe it might not let's see。
it's going to try to send I think it actually tries to send all of it at once here but but that's that。
and then after it sends all the data it needs to close the client okay where does that like what's。
the whole point what's the whole point here okay in this case let's see it's not blocking in the。
same sense we've let the we've let the kernel handle the events for us and in our case we are still。
able to do the extra work that we want to while the kernel is waiting around for data to either come in。
or not okay so it's not stopping us from handling this but when there isn't any data to send or。
receive we just that we do shut down we don't need to go back to that while loop so in some sense it's。
it's blocking in the sense that if there's nothing to be done our process turns off right but if we。
have anything to do either sending or receiving data it will be triggered by the fact that there。
is the kernel waiting telling us going oh you've got more stuff to send go send it you've got more。
stuff to receive you've got a client connection coming in do that so the the while loop will not just。
continue spinning like that it will actually block in the sense it'll block in more than one thing。
and that's really the big idea is that it's kind of blocking on both sending the data to all our。
different things if there's no data to send it will just wait around or if there's no accepting。
connections to accept does that make sense Amy to your to your wise not blocking anymore it's kind。
of blocking in the sense that it's it's not blocking but it's basically saying now we can wait for two。
things at once whether we have to send or receive not just wait and check again and again and again。
to see if we have connections so we're setting up these events to say we've got two events well。
we've got at least one event which is accepting clients and depending on how many clients request。
the data then we've got many more we've got other things to do as well every time we are able to send。
that the the e-pull will actually call the right function to do the the work for us so our our main。
function can just sit there and the we let the kernel tell us when to send and receive data that's。
the big difference so that's why it's not not blocking anymore we've leveraged the fact that the kernel。
will handle that waiting for us and our main threat doesn't need to do anything could we do more work。
yeah sure we can continue to do more work and now we're still waiting for two things at once but。
it's just abstracting it away a little bit that's all and I know there's a lot a lot going on here。
to see that and the day of the code you can go and read through the code to do that here's the bottom。
line okay the bottom line here is there are lots of ways to build a server okay we've seen a。
a number of different ways to do this okay we can build a threaded server we can build a non-threaded。
server and just kind of hope for the best we can build a threaded server that uses threads or uses。
thread pool to do that which does we do know that there are going to be some blocking calls。
and it's fine to do that again it's just maybe it's a different strategy to do that。
but we've now seen another one where you let the kernel tell you when data needs to be sent or。
received and then it will that'll work in a different way before I go on to more details let me show you。
what this actually looks like in this case this is the let's see we've got the。
efficient server I believe is what it is it's going to listen on port 3333 and if we go to what。
was it myth 61 again myth Smith there we go 61 333 it's going to serve us this website right it。
happened to be a website where I told that it went and pulled the image from somewhere and whatever。
but you would be able to do that many times and for every browser window that's open it will just。
feed a bit at a time and I think even if all of us in here tried to load that one page of the exact。
same time on our browsers you probably wouldn't see much of a slowdown because it's just sending one。
or two bytes of time but doing it very fast to all of you so that's how that that's that's what's going on。
there okay so where where does that leave us in the big picture when you're building a server or when。
you're doing anything and this is hopefully one of the big takeaways of this class it depends on your。
own system setup what operating system are you using maybe you're using an operating system。
doesn't support this event driven model well too bad all right Linux happens to support it right。
maybe you have a computer that has a really fast network card and there's less waiting you're not。
gonna do on that or maybe it's got multiple network cards or whatever what about the network itself。
what if you are trying to serve a website through like your phone's data plan right you might have a。
totally different idea of the best way to do that based on the actual platform so it's all about trade。
offs and it's all about how you can do that the big idea with the server except is and respond to。
as many connections as you can that's the big idea and respond as quickly as you can to client requests。
okay don't block the accept and don't serve too slowly that's those are the big ideas and that's。
the the basic idea of any server when you're setting it up okay you want to respond quickly。
and you want to accept and you don't want to block the the user okay um。
don't write a server that busy waits because we don't like busy waiting because it makes the fans on our。
computer turn on right we don't like that we're going to just wasted CPU time and energy and so。
forth okay but you have to understand some of these low level ideas when you're making this choice。
some of you will undoubtedly go to companies that do networking as their core business right。
some of you might end up in Cisco or some other company that has lots of networking right that's。
happening at their Amazon web services or something like that and you'll have to understand some of。
this low level stuff right why do you avoid busy waiting and and so forth and do you need to know。
all the details like all that stuff I just flew through well depends but you will if you get into。
that take CS 144 if you if you want to do that okay and you have to be smart about this too by the。
way right this is a great comic I love this comic because it it's one of the xkcd comics that basically。
it's this server that says hi I'm a server have you ever done this where you go to a web page on。
your web browser on your phone and the server basically gives you tries to give you the mobile。
version of the website but it doesn't take you to the right page takes you to like it's the main。
website and then you're like well this is exactly what's going on here right oh boy can I help let me。
get it oh you're a smartphone browser your browser tells the web server what kind of browser it is by。
the way and you can spoof it in some cases too you can pretend that you're a regular computer browser。
even though you're on your phone sometimes that's a good way to do it um I don't know if the main。
browsers let you do that now but they might you have to do sometimes you have to delete the end but。
it's more than that actually the reason the end shows up is because the browser your phone's browser。
says oh I'm a web I'm a phone right and the and the client or the server goes oh I better give you。
the paired down version even though your phones have regular full browsers on them so um anyway in。
this case this server says um hey I've got this new mobile version and uh the phone goes wait a。
minute you're giving me the main page and then the server just resets because it has no it's kind of。
done like that so build your servers smart if you're gonna build servers right don't try to be too。
fancy with them even though you've got new things like browsers and your watch browsers and so forth。
and it's uh it's their trade-offs but be smart about that okay all right so we are a little early。
which is great um as I said at the beginning of class you guys made it through a difficult class um。
you can argue whether this class is more or less difficult than CS 107 or CS 103 or whatever but。
if you weren't challenged by this class you're a smarter person than me because this is I think。
a challenging class you should give yourself a pat on the back for doing well and making it through。
all these crazy long assignments with 15 pages of reading and 100 files to look through and so forth。
if you want to take more systems classes there are tons of them if this is the last systems class。
you ever did well you should congratulate yourself on being done with that too。
and uh and go do your AI stuff and whatever um but uh and did I tell you about Chris Peach。
telling me that hey how important systems is to AI these days and so Chris Peach who you many of。
you know um does AI research and CS education AI research uh and he was talking about with me。
the other day about the fact that all of the AI that's done now uses GPUs and TensorFlow and all。
these kind of systemsy things to do AI and he says I wish I paid more attention in 110 in those。
systems classes or maybe I wish I wish I'd taken a parallel processing course so I could dig in a。
little bit more to figure out what's going on or at least I understood a little bit more about how。
this works so systems is not something that you necessarily you can do a lot of stuff without。
knowing much systems at all but you can definitely inform what you're doing in almost any field by。
understanding a little bit more about some of the decisions that are made and and so forth so uh so。
let's now I'll leave you with uh with that thought uh you guys are great thank you for bearing with me。
during my first quarter teaching this all alone I apologize if there were mistakes along the way。
but you guys are great I will see you on Monday at the final or during the office hours today and。
tomorrow and thank you very much。
