UCB-CS162-操作系统笔记-十-

UCB CS162 操作系统笔记（十）

P3：Lecture 3： Processes, System Calls, and Fork - RubatoTheEmber - BV1L541117gr

Welcome back everybody to lecture number three of CS162 and we're going to pick up where we left off。

And if you remember last time we talked about four fundamental OS concepts that help get us right away into sort of the meat of what operating systems do with respect to scheduling。

One was the idea of a thread， which was an execution context like a virtual CPU or processor。

which fully describes the program state， it's got a program counter registers， execution flag stack。

etc。 We talked about address spaces， both with and without translation。

And that's basically the set of memory addresses that are accessible to a program。

and they may be distinct from the memory space of the physical machine so in principle。

every address space can be a little different in terms of what happens when you go to the addresses。

but they'll all have the same kind of set of addresses typically。

A process combines those together to give an instance of a running program。

and it's really a protected address space and one or more threads and so we're going to work on that theme a little bit more today to give some more details and we also had some pretty good discussion on Piazza。

that you can take a look at from lecture two， and I'm sure that will continue for lecture three。

And then the final thing that we talked about which is kind of an essential hardware component that helps tie this all together is really dual mode operation。

where there's at least two modes system and user， mode， where one of the modes。

namely the system mode has more access than the other， which is the user mode。

and unless you have something like this， then it's very hard to actually put any sort of protection into the picture。

And so that dual mode operation is going to be very important as well。 Okay。

so the bottom line as we mentioned last time is really OS is run programs。

And remember a program is like， like a proto process or it's ready to run but not running yet。

And what you do is you take that program you typically generate it via some program source like in the C language or pick your favorite language。

It gets compiled and linked into that executable。 And then it can get loaded and become a process and that process is an executing program。

Okay。 All right， and this is this whole path here from writing the source。

compiling it and running it is something that you're going to get to do a lot of in this term as as we go on with the projects and homeworks。

And notice also that not only are we going to be able to play with user programs but we're going to be modifying the operating system as well。

And the other thing that we're going to just remove refresh your memory a little bit was this notion of a protected address space。

which is really something fairly simple。 It's the idea that the processor has a set of addresses。

which we're going to call virtual addresses that come out。 They go through some sort of translator。

And they become physical addresses and that's what's actually used to address the physical memory。

And this general idea of a translator is one that will show you a number of instances as the term goes on。

But just keep in mind that there's kind of the addresses the processor uses and the addresses that actually are physical and go to DRAM or some other storage media。

Those are different。 Okay。 Another thing we had last time。

just to remember where we were going was basically a picture like this that's kind of putting it all together and showing you what the process is。

And we can have more than one thread in a process。

So the simplest thing here is really a single threaded process。

And we can put more threads in as we wish。 Okay。 And the PCB is basically the process control block and we'll talk about that in more detail。

But if you notice here， just taking a single threaded process。

we can see that the thread kind of encapsulates the concurrency portion or the part that's actually running。

And the address space is kind of the storage or protection environment。

And every thread typically has registers and stack that are potentially stored in memory for when it's not running。

And if you look at this multi threaded case we actually have a bunch of for each thread we have registers and stack associated with that。

Okay。 And there were a lot of good questions about multi threading kind of on on Piazza as well。

And notice that this protection environment encapsulates all of the threads in a multi threaded environment。

So we have， here we have three threads， they share the same protection domain。

So that means they can override each other by accident。

But the reason we want them in the same protection domain is it's much easier for them to share information。

And if we really were worried about maliciousness or overwriting we would create a separate process with its own threads。

So within a process the threads all share the same memory between processes they're protected from one another。

Okay。 And it's that protection which we're going to talk more about how to get today。

So there's another good question so these threads and say the multi threaded one are they all part of the same application。

Yes。 They're all part they were all generated by the same writer or they were linked together， etc。

And so different processes can be different applications。

We're also going to talk at the end of this lecture about how to have more than one process for application。

But you never have more than one application per process。 Okay。

Because that's the protection domain。 Now， if you remember。

we started this whole process of understanding address spaces and translation with this very simple thing I call base and bound translation。

And the idea here is that on disk or the image of the program kind of has code and data and is going to need room for heap and stack。

And we can think of it as starting at address zero as it's linked。 And when we start running it。

we can have our physical memory may have many instances of the same program or may have different programs。

And they're all going to have different chunks within the memory that sort of store their data。

And notice I put in gray here this is also the kernel， or the operating system has its own。 Okay。

And the， in this particular instance， what we notice here is that because we put this yellow portion or the program that becomes a process here。

starting at one zero zero and memory。 It will no longer be capable of running from zero unless we do something and that's where this translation comes into play。

So even when the processor talks about an address like zero zero one zero。

which would make sense over here， because we have a base and bound base red address and bound。

we actually add the base to what the processor does。 And the net result is a translated。

a translated address， which is appropriate for where it is physically so if you look here the CPU says。

zero zero one zero， which is somewhere in the static data。 The data adder。

which is in hardware adds the base address and as a result that addresses make sense also in physical memory。

Now there's a question here， several questions that are kind of interesting one yeah gray is the operating system。

So two is can there be protection without translation and the。

and yeah absolutely yes so we the first version of base and bound that we showed you last last。

week on Thursday was one without the little plus here and all it did was just made sure that the addresses that the program had were between base and no farther above base than bound。

And that means that when you took stuff out of disk you had to actually do the translation with a dynamic linker before it got put into memory。

Once we put this dynamic translation into the picture then the CPU can continue using the addresses it thinks are there。

which are relative to the image and the hardware translates it for you。 Okay。

now the question of the base and bounds always use a constant offset the answer is no。

the operating system loads this yellow thing in there and then it sets the base address for where it loaded it it sets the bound address for how big it is。

And we could， you know we could put another copy of the yellow thing here in a different part of memory and when it's running we put a different base and bound。

Okay， all right， now we will get back。 There's a good question here about the OS preventing a denial of service attack from malicious process that spawns many threads。

So that's an interesting question， but it's perhaps thinking in the wrong way here so all of the threads in a process are part of that。

application and they're written by one person so if you're launching too many threads you're basically executing denial of service against yourself and that's called a bug。

Okay， now， notice also that because of this base and bound。

the program running or the CPU excuse me has no way to address anything below so it can address gray or above so it can't go below because that would be a negative address which the CPU doesn't do。

and it can't be above because if the CPU tries to use an address that would exceed the bound。

then it's faulted and so this what we've shown you here basically protects against this program touching anything that's outside of its bounds。

Okay， now let's take a look briefly。 I started down this path at the very end of the lecture last time and I wanted to finish up what we were talking about just to go through it。

So if you notice， for instance， here we have the yellow program， which is now a process running。

We have another one which is idle is green。 We have the kernel which is idle which is gray so right now let's assume again I said last time that there's only one core in the whole system or one CPU。

And in that instance when we're running in this user code we're not running anywhere else。

And so these registers here， which represent registers on the CPU notice that the program counter is pointing somewhere in the code。

And the stack pointer is pointing somewhere on the stack。

and we have basin bound set up here for starting at one zero zero and and then the maximum kind of bound as well。

And as a result this program is happily running away in that yellow area。

And notice that we have a system mode bit here which is zero saying that we're not in system mode and so we're at user mode。

And so this current execution environment is one in which if the code were buggy and it tried to address below or above where it's supposed to。

There would be a fault and the operating system would take over。 Okay。 So。

of course as you might imagine， if we can't ever get out of yellow then we can't run the operating system we can't ever run any other process so clearly there's got to be a way to get out of yellow and into the kernel and we'll talk a lot more about。

either voluntarily yielding or timer interrupts etc as we go on。

but really the question is assuming one of those events happens how do we go from this environment to one say where the kernel is running。

And you can imagine what needs to happen so one we're going to have a system mode go to one。

which is going to mean that the basin bound get ignored and now the kernel is going to have control of everything。

So that's one thing。 And the second is going to be that we're going to need to save aside the PC and the stack pointer and other registers that are currently being used by the user program we need to save that so that we can restore it later。

Okay。 And notice also that we're all we're going to have to save them aside but then simultaneously go and start running in the kernel。

And so here we go so we're going to assume for instance and interrupt just happened and take a look at what's a little different here so the basin bound。

which represent this yellow thing are not in use because we're in system mode one and so it basically is allowing the full set of addresses to be touched。

The other thing is that the PC that was running is stored now in the user PC register so this is like a register set aside for right after an interrupt to keep track of where we came from。

Okay， and we have the new interrupt code which is going to be an interrupt vector is going to take us to the operating system。

And assuming we do all this cleanly at this point now the operating system can kind of save all the other registers for the yellow and then maybe allow the green to run。

Okay。 Now， there's questions here about you know if virtual memory space has everything in it then how can you ever have a segmentation fault。

And basically we'll talk more about that as we go but here's an instance where there's a segment we're running in。

which is a restricted size and so if you try to go outside of that you get a segmentation fault but it means to have the illusion of infinite memory is the operating system could choose to make the segment bigger。

Okay。 Now this interrupt vector is actually not storing any state what it is is it's storing the pointer to whatever interrupt occurred so that could be a disk has data that came back and so the part of the interrupt happening is that we're going to start running in a special part of the。

kernel to handle that interrupt。 Okay。 Now， how do we save the registers and set up the system stack for running well first of all we take all the registers we save them off in memory somewhere。

Okay。 And this the kernel can do that because it has we're in system mode and so it has access to all the registers and be。

you know， we have access to all the memory and so we can then take and load pre-systems。

And load preload the green process and get it ready to go so notice that what I've done is I found so this is the process control block we mentioned earlier for yellow we find the process control block for green。

We load it up。 We put the base in bound for green we put the user PC we set up the stack and then what we're going to do is we're going to do that return to user mode and notice by the way that the。

C。V。 is actually stored in the static data of the kernel we'll talk more about that as we go。

And when we do a return to user poof all of a sudden we're in system mode zero。

the program counter got a transfer from the user PC into the program counter and we're now running in the users code。

the base in bound are active and the stack is active and now this process gets to run。

as if it has exclusive control of the machine。 Okay。 Now。

basically the question here where do you save the PC value of interrupts from different processes。

So let's not get too confused here so right now when it interrupt occurs it's going to take a kernel stack and start running on it。

And it can be the kernel stack associated with the current user process etc。 We'll get there。 Okay。

so just suspend that understand or that question for a little bit and ask me in maybe a lecture or so。

Okay。 But for now just this understanding here that we took if you look back here right after the interrupt occurred。

say it's a timer interrupt， we have the users PC which the kernel can now save out to the PCV can load the PCV for green and we're good to go。

Okay。 Now， so here we now have resumed and we're now running green。 So。

are there any questions any more questions on this。

So what I've just shown you here is this simple idea we were running yellow and interrupt occurs。

We run the scheduling code to now run save yellow run green and we're going to go back and forth with that and that's going to basically give us the illusion that these two yellow and green are running at the same time。

Okay。 Right， and that's good。 Maybe I misunderstood that yes every process has its own PCB。

Now， so how do we run many programs well we have the basic mechanism now to switch between user processes in the kernel we just showed you go from yellow to green and then same thing would work from green to yellow。

And the kernel can switch among user processes and protect the operating system from user processes and processes from each other。

That's that protection domain I showed you with branch and bound right the basic mechanism of branch and bound really protects。

All of the user processes from each other and from and from messing up the kernel。

The kernel is in charge of everything and so the kernel。

If you break into the kernel in this basic model here you can。

you can trash everything but we're going to assume for a moment that the， kernel is not broken into。

Okay。 So we have a lot of questions here one of which I will answer right away which is what if they don't fit in memory。

That's a great question。 Right now this model is not great if if the yellow or the green try to grow too big。

So we're going to run into issues here。 Okay， and because we're requiring the whole program to fit in one contiguous bit of memory so that's why we're going to very quickly discard branch and bound as or base and bound excuse me as our desired mechanism here。

Okay。 The thing that switches between user and kernel mode is the hardware。

That's what the processor does when it receives an interrupt。

It is the thing that does that switching。 Okay， and it's the thing that's doing that dual mode switching from user mode to system mode。

How do we represent user processes in the OS we mentioned everyone's got a PCB。

How do we decide which user process to run。 Well that's a good question that's called scheduling and we're going to actually have a whole unit on scheduling coming up。

But just because we can switch between green and yellow and yellow and green。

Doesn't tell us anything about when we switch or how much time we give each of them。 That's a。

that's a whole another discussion that's actually pretty interesting because there's many reasons you might have for giving one more time than the other or interrupting it immediately or whatever。

We're going to talk about that a lot。 Coming up later。

How do we pack up the process and set it aside well we just showed you we basically unloaded its registers into its PCB。

How do we get a stack and heat for the kernel well that's interesting we'll have to talk about some of that。

Memory allocation。 Doesn't look like we're wasting a lot of memory by forcing kind of all of the user programs to be loaded at once。

Certainly。 So we're going to have to start talking about swapping and then ultimately we're going to talk about paging which is even cleaner。

But so the process control block is an essential piece and you know they're different for every operating system as to exactly what the format of them are but you get the idea what's in there。

It's got some state of the process。 It's got， you know。

a number for the process ID and maybe knows what the counter is and some of the other registers。

Memory limits， etc。 All of that's in the process control block and the kernel uses that to track the processes because especially in the next couple of weeks where we're mostly talking about one core or one CPU for the whole machine。

Most of the processes are idle and in stored in process control blocks。

There's only one of them that can be running at any given time。 Okay。 Now。

the kernel scheduler is this thing that we're going to talk more about for basically deciding which gets swapped in。

And we can give non CPU resources， memory， IO， etc。 Also by a scheduler。 And I would say。

as we've said a couple of times now the process control block there's one per process so everyone has。

Every process has a separate process control block and they're stored in the kernel memory。 Okay。

Now， let's not worry about maliciousness for now。 The user to kernel mode switches are under control of the kernel and we'll talk a lot more about maliciousness when we go much later in the term。

But let's assume that where the right things are happening so far。

So what we just shown you here is pretty simple。 So this is two processes one core。

It sounds like sounds like an ad campaign right two processes one core。 We start with process zero。

which is a user mode thing。 Okay。 And it starts executing。

And if you notice right at about the end of this blue arrow， there's something like an interrupt。

Okay。 And at that point， we're going to save enough state of the process and we're going to enter system mode。

And the system mode is really running kernel code and that's the point at which we save a bunch of stuff。

And we load we save a bunch of stuff into PCB for zero and we load it back from PCB for one。

And then we're going to start running process one， which is also going to be running in user mode。

Okay。 And， and notice， so the question about who hands out non CPU resources。

So that's going to be a combination of the scheduler in some instances and other parts of the kernel。

So we'll definitely get to other resources later。 But let's stick with things simple things for now。

Okay。 And so if you look what you see here is process zero runs for a while。 There's an interrupt。

We saved state we really reload state。 We start process zero back in user mode。

We run for a little while we come back， save state for process one。

And then we sort of rinse and repeat based on scheduling。

And if you notice these little places that I have yellow circles actually represent a user to kernel or kernel to user transitions。

So these are the points at which we actually have the change in the system mode。 Okay。

And the other thing to note is what's in red here in this particular diagram is 100% overhead。

Right。 So there was a question earlier about what， what about the overhead is switching。 Well。

here's overhead。 Okay。 And that's basically the time to handle the interrupt and save and restore registers and then make scheduling decisions。

That's all overhead。 And so one thing that you can start thinking early in the game。

Is that if we switch back and forth too frequently。

we're going to actually end up where we do more switching than we do a computing and that seems like a bad idea。

Right。 So there's a minimum amount of time that I'm going to throw out a number there。

But oftentimes if it's bigger than 10% people will say you're thrashing。

Don't worry about those exact numbers， but if you're wasting more than 10% of your CPU cycles in the switching process it's probably bad。

Okay。 And when I said kernel memory I meant the great part earlier。 Okay。

So what's running in here in between the save and reload。 Well。

there's a decision about what to run next and that's a scheduler。

which I'm not going to go into great detail today。

But think of this as a continuous loop where it says， well， I'm going to schedule。

So if there's a ready process pick one， run it for a while， and then loop back and do it again。

Okay， and this arc is going to be triggered by something like a timer or a yield or I/O。

And we'll talk a lot about that next time。 Okay。 But scheduling。 What about scheduling？

It's this select process thing is a mechanism for deciding which get to run next which processes are threads and lots of different scheduling policies。

So we'll have ones that give you fairness。 We'll have ones that give you real time guarantees like might be in your Tesla when you slam on the brakes。

You know， there's some real time guarantees that better come into play or it runs into something right。

Maybe there are some optimizations for latency。 These are all interesting scheduling policies。

And so you'll be many different ways。 Okay， to think about it。

There's a good question kind of in the chat about how the OS knows which registers to save and restore。

And that's going to be that's going to depend on kind of which registers are active。 And so。

mostly it's a decision to just save them all unless there's more information。

You're going to potentially get the potential you're going to find out in project one that if there's。

for instance， an FPU running， then you have to save and restore the FPU registers。 Okay。

And we want to again just reiterate that we don't want to be spending a bunch of time saving and restoring or scheduling。

And so we do not want to be going back and forth too often and again keep that 10% number in mind。

Okay， so I， there was discussion about this in Piazza and I thought I'd bring this up。

There is this notion of simultaneous multi threading or hyper threading。

This leave it to Intel to take something that people knew already。

which is simultaneous multi threading and give it a different name， which is hyper threading。

But the idea is that the hardware， it's a hardware scheduling technique that basically allows super scalar processors。

which are processors that can run more than one operation at a time。

To have two simultaneous threads or three threads or fourth threads running at the same time in the same pipeline。

And we're not going to spend a lot of time talking about this because that's for 152。

But if you look， here's a single CPU where each think about time going down here and solid blocks as opposed to gray represent cycles where actual executions happening。

So here， notice that in with only one thread we're wasting all of this gray time in the pipeline。

But if we put two threads in the same pipeline， then we can fill things in。 And you could say， well。

here's two pipelines。 Here's actually hyper threading。 So this would be two cores。

This would be one core with two hyper threads。 The two hyper threads fill out and better utilize the pipeline。

The difference between two hyper threaded cores and two cores is small from an operating system standpoint。

It's much bigger from an architecture standpoint。 And we're still only talking for the next couple of weeks about one CPU。

one hardware thread。 But you'll be able to see when we start having more cores that more cores and hyper threads are pretty close to the same thing for you。

And this scheduling that I talked about in software earlier is actually happening in hardware。

And it's happening potentially on a per cycle basis。

So you can have two different threads on the same cycle。 Now， also remember， and again。

not for the next couple of lectures， but remember the world is parallel。

So even at the single core level， you can have something like this Skylake from 2007。

and then there's a more recent version three of the Xeon processors from Intel。 Lots of cores。

lots of hyper threads。 There's two 28 cores， 56 threads is one example of Skylake。

Lots of parallelism。 Okay， so even though we're talking about how to deal with one core。

once we figure that out， then it'll actually be pretty clear。

hopefully to everybody how to do multiple cores。 It's not going to be a big deal。

Okay。 Yes， and we can， you guys can talk to me offline about policies in hardware in hyper threads。

but there's all sorts of policies about who gets to go when there's not enough resources。

Okay， so is the base in bound。 I said branch in bound there。 Sorry。

A good enough protection mechanism。 It's base in bound。 And the answer is no。

it's really too simplistic for real systems。 Okay。 And it's inflexible or wasteful。

You have to dedicate physical memory for potential future use。

So if you want to have enough space where the stack can grow。

then you got to allocate a huge big chunk just for that。 Okay。 So it's also got fragmentation。

So if you have processes with a bunch of different sizes， and you want to have， you know。

them exit and then bring in new ones， you're going to very rapidly get fragmentation。

And you're going to have to start moving things around in physical memory just to make just to find space to fit。

Okay。 And so this idea of the base in bound is really not great as a sole mechanism for translation or protection。

It's even worse when you think about sharing。 So picture your mind back to that yellow and green block。

If those two processes want to directly share memory。

there's no mechanism in the current base in bound that I shown you for that to happen。 Okay。 So yes。

the OS can resize the bound to give more memory， but there's no way for yellow to talk to green and green talk to yellow because by design。

they don't， they can't overlap each other。

Okay。 And so what's one clear thing that we're going to do when we get into virtual memory and greater detail is we're going to start talking about a generalization of base in bound。

which is the segment idea。 And here， for instance， in the x86， which is the processor。

you're going to become very familiar with in this class。

There are multiple segments like a code segment and a stack segment。

And each of those segments has a base and a bound。 Okay。

and we can have multiple segments and two segments from different processes could actually overlap in physical memory if that was our choice。

And now we've got a way to have them communicate through memory。 Okay。 Now。

this is one thing we could do， but even better is actually paging which we talked about last time we started talking about it and you heard about this in 61 C as well。

And then the idea is that you take the code data heap and stack of each process。

And everyone in the addresses goes through a generalized translation unit。

and can be pointed at in pieces to all sorts of pages in the physical memory。

And then we can get a translation on page size where all the pages are fixed。

And that fixed size like 4K or 16 K is going to prevent fragmentation because every page is the same。

And we can get more stack or heat just by adding a few more pages to the translation map。

And that's where paging comes into play。 And modern virtual memory is almost exclusively page based。

And there's a lot more about that in greater detail。 Okay。

All right。 So， time to get really started with the class。 If you haven't gotten started。

I will tell you that homework zero is due tomorrow。 Okay。 And so。

I'll put up some other office hours。 I apologize。 But homework zeros do tomorrow。 Okay。

And so you got to get familiar with 162 tools configure your VM submit the get all of these things。

And then we have homework zero do quickly is to get you guys all on board because these are all things that you need to know how to do right off the bat。

Okay。 And you should be going to some section now。 It doesn't really matter which one。

because there's sort of crucial other information we want you to get a whole of in terms of understanding。

you know， C language and so on。 And I hope that everybody who everybody got a chance to go to the sea review that we had on Saturday。

I believe that if you didn't go there's videos of it， you should all review that because again。

just like I was mentioning with homework zero there's a bunch of things we assume。

that you're going to know and be ready to go with。 Okay。

And those are kind of like baseline before we really plow in。

And don't don't get too worried about this in the sense of how can I possibly do this。

We understand that 61 C left you with a very basic understanding of C。

but that's why we give you these resources in the first two weeks so that you can really get up and running。

Okay。 And so take that as a challenge to get good enough for C that you can start doing the class。

Okay。 And I'll put up， I'll put up better office hours I realized I had these earlier today and I realized it's not going to work so I'll have better office hours for you。

The other thing that's really important is this。 Friday is the drop deadline。 Okay。

so that means that you have to make a decision to drop otherwise it gets tricky because you start using up your。

you know， once a career kind of drops or whatever。 It's available。

And if you know you're going to drop you should do that now because that way we can bring more people in from the wait list。

And the other reason we have an early drop deadline is because when we form our groups next week。

we want to make sure that we have groups of people that are going to stick together and so we don't have orphan groups where there's a couple of people are all that's left。

Okay， and so really do a careful thinking about do you believe that you want to keep up with the class。

then please do because we want to have you here。 In fact。

I think as of right now we may have led everybody in off the wait list so that could come to the class。

But if you're not going to make it， please make a decision to drop Friday。 Okay。

And if you know of any friends of yours that have stopped coming and haven't dropped。

let them know that they need to drop because they will get put into the class。

And then I have this every semester I have somebody who doesn't realize they're still in the class until the very end of the term。

And it just， it's a mess。 So， I'm sure all I'm glad to know that all the people that are。

that are here are excited about the class。 And I know somebody who might still be on our on our class list and it's actually planning to drop and might be in danger for getting。

Please make sure that they don't do that。 So the group registration form stuff's going to come out over the weekend and or next Monday so really it's next week that we start really getting our groups together。

And that's going to be the an autograter form or something of that nature and groups are for in a group。

Okay， you know， that will not have three in your group or add another to make five so make sure that you start looking at four people。

And find a group。 All right， and we'll help you if it turns out you can't but it's it's often much better to have folks that you know in the group。

Okay。 If you only have a group of two， you might post on Piazza。

There's lots of lots of people up there we have a special kind of thread about people looking for group members。

So try to find somebody yourself。 I mean we'll help you a bit but might not hurt to try。

try yourself first。 Okay。 And you're going to want to put down a section。 And then suggestions。

Okay， we'll have the ability for you to ask for a couple of sections。

but you want to try to make sure that all of you in your group can attend either the same section。

which is ideal or at least the same TA。 So we have some 20 hour TAs that have two sections。

All right。 The midterm， the day and times are fixed now for midterms one， two and three。

I believe they're all the seven to nine PM and we'll make sure that everything's updated。

And there is， there's going to be a conflict form to figure out how to deal with conflicts。 Okay。

Now I'm going to say this again about discussion sections if you look about the first bullet here。

We don't have discussion sections permanent ones until you get your group because your group all has to be in the same section。

All right。 So for this last week and this week you can attend any section。

but starting next week you need to pick a set of sections that we'll choose from in which your group can all attend the same section。

And it works in worst case you and your group are in this with the same TA in different sections。

Okay。 We， I think I've answered all of those questions。 Are there any others？ Okay。 Yeah。

you know this class get started pretty quickly。 But part of the reason we started so quickly is because there are some things where we want to make sure you're really rare and to go and ready to dive in and be productive。

Yeah， so please， as Anthony just said in the chat。

please try to find your own group rather than us randomly assigning。

Now there was a one question in chat that I noticed that I wanted to talk about briefly and it was a good one。

So the question was， can why don't you page。

Why don't you combine paging with segments。 And the answer is the way we've described segments right now it isn't a paging scenario because it's got a base and a bound。

Okay， and so that unless you made every segment exactly the same size it wouldn't be good toward paging。

Right， however， as we'll show you is you can easily combine segmentation and paging and in fact the x86 actually does that。

But that's that's for a lecture in a few weeks。 Okay。

but for now basic segments that we've described to you don't have paging potential because they're variable length。

Okay。 Now。 So if you remember， move and light along。

We talked last time at the very end about there being three ways to get from the user into the kernel。

Okay， there are three types of kernel mode transfer。

And I want to I want to explore that a little bit more so one is a system call。

And a system call is going to be the basic way in which user code can ask the kernel for facilities or services like file systems or communication across the network。

Okay， and so this is an example where a process requests the system service。

It's kind of like a function call but that function calls actually going to go from user mode into the kernel and then back out again so it's a function call。

but it actually goes through different modes。 And you're going to get in project one you're going to actually get a chance to write some system call code。

Okay， so you'll be familiar with this as you go forward。

This is going to also be some similar to what we're going to call a remote procedure call later。

because you're going to take a bunch of things that were part of the function call and they're going to get packaged up in a way the kernel can accept them。

And then they'll get returned back later。 Okay。 Now， many system calls。 Okay。

we're going to show you a bunch of them。 As we go over time， there's a question here can you know。

can a process ask for more processing time well certainly in some schedulers there's a way to ask for more time。

There's certainly ways for a process to say hey I'm done for now。 You can let somebody else run。

There are many， many system calls okay and I think in the latest versions of Linux there's thousands of system calls。

At some point that API gets to be a little excessive。

Another way other than a system call which is a voluntary request from a user into the kernel。

We also have this idea of interrupts and interrupts are external asynchronous events that force the system to go from user mode into the kernel to handle an event。

And it's totally independent of whatever the user was doing。

So the user is busy computing pie to the last digit and interrupt might come along it's going to suspend and go into the kernel。

handle the interrupt and then go back to where it left off computing pie。

And why do we have interrupts well interrupts are for things that are unpredictable or unknown like when is a packet going to come in from the network or when is the disk going to have its data back for you。

And a very important interrupt which is going to be very important next time when we talk more about scheduling is going to be a timer interrupt。

which is how the kernel can be sure that it can steal time back away from the user。

So if we were switching between the yellow and the green and the yellow and the green。

The way we make sure that the kernel can always do that is a timer interrupt goes off。

And if yellow is running， we go into the kernel the kernel will change things around and then let green run and then a timer will come off and then it'll let yellow run。

And so timers are going to be kind of one of our key mechanisms for ensuring fairness of execution between different processes。

And then another one is a synchronous exception or trap。

And these are examples in which something that's tied to what the user is doing can't go forward。

The simplest example is an easy one divide by zero so the users run in some code they try to divide by zero。

The answer is big， right， very big。 So that's an exception that can't continue。

We enter the kernel with an exception。 Another good one that you're going to see a lot of a little later in the term is a page fault。

So the user tries to use a part of the address space that doesn't have any physical memory backing in。

Well， we'll get a page fault exception that'll be in the kernel and the kernel will then make a decision about how to handle that or whether to boot the process out of the picture there and kill it。

Okay。

Now if you remember， I showed you kind of this new last time to kind of the circles where we had the hardware was the brick underneath and then we had kernel mode which was things that handled hardware directly and then user mode which kind of floated on top of everything。

And so at some point we have an exec call where the kernel has been asked to start a brand new program running that exec call transfers into user mode。

And now we're running a program as a process。 And during the course of the execution will maybe make service calls of the kernel but eventually we'll have an exit which will take us back to the kernel and this is the life of a process from point A to point B。

Okay， and we're definitely going to cover scheduling in a much greater detail in later lectures。

But for now let's look at the mechanism of getting things running and what happens when they're running。

And so when you're running， for one thing a user program might make a system call like it needs to read from disk。

So in that case that system call is in the kernel it might actually return right away。 Or。

and after it's returned it might actually have the kernel start something in the hardware。 Okay。

and in that instance later we might have an interrupt that will pull us from user mode into kernel mode。

And that interrupt may cause the kernel to go mess with the hardware to read something out of the hardware like some network packet。

The result might come back， might do multiple reads eventually we're done。

We restore the user's state returned from interrupt and poof we're now running user code again。

And so notice that all of these transitions out of user mode into the kernel mode are done in a way that are going to make sure that the user mode can continue running where it was without interruption or without it's state being destroyed。

And so it can continue as if the kernel never took over。 Okay。

and that's part of that saving and restoring of registers which we're going to have to make sure we understand there。

Okay。 Now， the question here， can we assume that libc functions are usually going to trigger a context which not necessarily okay there's a lot of things if you look at this system call。

There are a lot of cases where a system call goes in the kernel and asks for information and comes immediately out without any context switch。

And so， and then libc is often a shield over the system call so when you're calling these library functions it may make system call it doesn't necessarily mean that there'll be a context which。

Okay。 Here's an example where that user mode might divide by zero in which case we enter the kernel and the process is done。

But other than that we can go back and forth for multiple system calls interrupts etc。

And then eventually we exit。 Now， how do we make this safe。

What do I mean by safe。 What I mean by safe is the user code's running。 And， you know。

we go to the kernel to do services and stuff but we want to come back to the user mode in a way that the user program can continue properly so that's one part of safe。

And the other part is really going to be that there's nothing the user code can do to screw up the kernel。

Okay， so there's a lot of different uses of safe here。 But for instance。

you know it shouldn't be possible for the user code to make a system call with bad arguments that will cause the kernel to crash。

Okay， or whatever there should be no way for the user mode to screw up the kernel mode。

So there's a couple different notions of safe here。 And in going from user code to kernel mode。

We have to make sure that all transfers into the kernel are controlled。

Okay， and what is controlled mean controlled means that no matter how screwed up the request is from the user。

it shouldn't be possible for the user to pop into bad part of code or anything like that。

It should be controlled as to exactly what part of the kernel code runs that you always start at the beginning of kernel functions。

etc。 All of that needs to happen。 Otherwise malicious user code could actually screw up the kernel。

And that's going to be part of what we talk about here。 But among other things。

there needs to be a separate kernel stack to make sure that whatever the kernel tries to do isn't impacted by how the user has screwed up their own stack。

Now a stack is a concept that you've got from 61 C and we're actually going to remind you a lot more about it next time。

But if you remember， it's that thing that makes sure that we can have recursive routines where the local variables are stored on the stack and we can push down the stack and pull it up。

So， and in addition to a separate kernel stack， we have to make sure that the kernel code is careful to pack up everything the user is in the user's request and kind of validate it and make sure that it's correct for one thing and pack up all the users registers and so on so it can be restored。

And it should really be impossible like I said for a buggy or malicious user program to cause the kernel to corrupt itself。

So this idea that the user is asking the kernel to do things is a nice notion because the kernel has got control of everything。

but the kernel has got to be careful and do it and allow that in a way that the user can't screw it up。

Okay， and we're going to this is going to be a theme。

this idea of security and protection against malicious things it's going to be a theme that we're going to work on as the term goes on。

Okay。 Now， let's talk about hardware support here we brought that up earlier like what who has the the actual system mode bit and who does those transitions that we talked about earlier。

And so that's the hardware， this idea that an interrupt like a timer interrupt can always take control back from a user program is in a similar P in a similar vein is about interrupt control。

And so interrupt processing as we mentioned earlier is not actually visible to the user process for one thing。

And it's always can always be guaranteed to happen。 Okay。

so that you could kind of think from the user standpoint that the interrupt happened between two instructions。

instruction a was happening the interrupt occurs bunch stuff happens。 So。

what is the state instruction B happens from the user's code。

they don't notice the difference except for what so what what one thing can the user notice to know that interrupts have happened anybody figured that out。

Exactly。 Okay， lag time exactly so that's going to be an interesting thing because in theory we're virtualizing the execution but there are these periods where time goes away。

And that's going to be a theme a little bit later。 So the interrupt handler。

what's a handler a handler is the thing that is invoked inside the kernel on behalf of of that interrupt and so if it's a timer interrupt。

We make sure that that timer interrupt is the right place in the kernel and that interrupts are disabled so we don't have this permanent interrupt interrupt interrupt interrupt until the timer interrupt can actually take over and execute long enough to clear the interrupt field。

Okay， now the question here about no context which is there's no context switch in the case of this interrupt happening in the what a context which is means to us and is going to mean a lot more to you next time is really a context which is when one user process gets switched out for another one。

or one thread gets switched out for another one。 So that's the that's the thing that's getting switched。

Okay， and basically what that interrupt handler is going to do is pack up any thing that can't be handled immediately into an OS thread for later work。

Okay。 And a good question about what happens to interrupts when interrupts are disabled is they typically just sit around until they get re enabled and then they take over。

Okay。 And if we have OS threads then potentially the scheduler is indeed going to schedule OS threads。

Okay。 So think of OS threads as a， if they're like user threads but they don't have a user aspect to them。

We'll get more of that next time。

So one of the things just to give you a little flavor for what interrupts are and don't worry about this too much but every machine has typically what's called an interrupt controller。

So it's not a CPU or our core and there's typically an interrupt disabled bit。 Okay。

And then the interrupt controller basically is a place a chip piece of hardware where all the interrupt lines that are external including the timer interrupt。

Go through a mask which decides which of them are going to be enabled and which aren't。

And a priority encoder to decide if multiple of them are on and enabled which one's important。

And if anyone is enabled and its priorities been chosen then the interrupt number gets sent to the CPU。

And at that point it's going to take an interrupt save the state and continue。 Okay。

And so interrupts are invoked with interrupt lines typically from devices and interrupt controller basically chooses the interrupt request to honor based on how it set the mask and priority。

And the CPU is basically going to go ahead and start running that interrupt controller。

And most again we'll talk a much more detail when we get into scheduling。

But most interrupt situations are such that as soon as the interrupt occurs this disable every interrupt bit is set long enough for the CPU to do things with the mask and so on to avoid that infinite recursion of interrupts。

And then once it's sort of fixed that it can turn interrupts back on to allow others to occur。 Okay。

And the CPU can disable all of them with this interrupt flag。 Right。

And that's a topic for a little more detail。 But I wanted to give you a flavor for the hardware context here。

There's basically these actual physical lines that say hi I'm a network there's a packet ready。

And that physical line causes an interrupt to occur which will take us out of user mode and into an interrupt handler。

Okay。 Now the question about non blocking on the chat there is basically running to completion。

Okay。 We'll be much more detailed about that as we go forward。

And there's also this non maskable interrupt idea which is an interrupt that can't be turned off under any circumstances。

And often is for things like， well your power is failing and I've got a couple of seconds worth of capacitance to hold me up better do something it's an emergency。

Okay。 So let's look at this idea here that let's suppose that the network is causing an interrupt and it's interrupt number five。

What happens？ Well that number five gets into the CPU。

And then that number five is used to look up in an interrupt vector which is a chunk of memory in that's accessible in the kernel only to the kernel。

And number five one two three four five has a slot in it that points to the interrupt handler for interrupt five。

Okay。 And so notice what's interesting this is a very good example what I mean by controlled entry into the kernel。

We use a number five and a table to make sure that we go exactly to the right interrupt handler and to the beginning of it not somewhere in the middle。

Okay。 So the fact that we use an interrupt vector and a table means the kernel can have very carefully controlled code that's been compiled and looked at millions of times that will be invoked whenever say a network interrupt occurs。

Okay。 And that's part of the key of making sure that the kernel it's running with really high priority can actually run only that piece of the code that has been vetted to properly handle say a network interrupt。

Okay。 And we're going to see the same idea with a system call。 Okay。

And I'll say it here just briefly we'll talk about system calls in a second。

But when a user makes a system call， like I want to read a file that system call the way it goes from user to kernel is with a system call number。

And that number will look up in a vector table and it simultaneously with going from user mode to system mode。

It will also make a decision about which of these vectors to run。

And so the user can't just say yeah put it in kernel mode and start running at this address。

because if they did that that would be a security violation instead they have to say， hey。

I want you to run system， call 12 for me。 And in hardware it will simultaneously save the user's PC transform into system mode and start running the vector 12。

And by doing that atomically we get our nice controlled entry。 Okay。 All right。

The question about is there some redundancy here where different handlers might。

The same handler might be in different slots I think was the question yes under some circumstances that will happen。

Depends because there is a way typically of reading the idea of what interrupt number you've got as well。

And so there are many different ways to structure this and we'll look at at least one of them in Pintos。

All right。 Now， where do you see this this dispatch pattern well it's in system calls it's in exceptions。

etc。 And we take interrupts safely well we just talked about the interrupt vector so it's a limited number of entry points。

There's the kernel stack， which means when the interrupt happens we just set aside everything the user is doing。

And we reload the processor with a kernel stack that's known to be good。 All right。

so that no matter what the user was doing， excuse me， the kernel is in a good state。

Potentially with interrupts we got to turn that interrupt that just occurred off so that we don't get an infinite recursion。

And then we have this atomic transfer of control。 Which in a single non interruptable chain we get we get a new program counter a new stack pointer。

We set the memory production protection properly and we make a user kernel mode switch and we do that atomically so that we go from the user to the kernel in a very controlled way。

All right， and that's how we're going to be able to make sure that the kernel gets entered cleanly。

Okay， and then of course we want to make sure that this is transparent to the user and restartable。

And so， even as we make an atomic transfer into the kernel we need to save everything that the user was doing。

In a way that allows us to restore it so that we can come back from the interrupt and the user can pick up where they left off with their computation kind of unscathed in that。

Okay， and so this idea of a separate kernel stack is just I wanted to say a little bit more and then next time and the time after we'll show you in even more detail because you're going to get deep into the kernel in a bit。

But we can't use the user's stack。 Can anybody figure out why we wouldn't want to use the users。

the memory that the user's stack point register is pointing it。 Yeah。

I like that safety and cap security and lowercase I don't know if there's a difference between those two。

But for instance the simplest thing is they could have a bad address in their stack pointer。

And if we use it the whole machine could crash right so really we have everything for the first month or more of the class is going to have this two stack model which we're going to talk in great detail so don't worry about it too much。

But every user program or process that's running has a kernel stack associated with it explicitly。

And when we're running the program itself。 We were up in the user code and it's got its own stack that it's running。

And if the process is suspended and sleeping， because say green is running so yellow is asleep。

If you look at yellow's resources， there's going to be the yellow's user stack but even the yellow's kernel stack is going to have the saved state on it。

And going to be suspended while the green is running。

And then if we look at what happens when we're we've done a system call into the kernel。

notice that every time we make this transfer from user to system we get a new kernel stack。

And then potentially if the system call requires a bunch of stuff to be done in the kernel。

we use the kernel stack to do that stuff。 Okay， and so we have a nice clean guaranteed stack it's small。

Okay， Pintas is going to be 4k and we'll talk more about that。

But it's enough to run what we need down here and then when we're ready to return from interrupt in particular case。

we return all of the stuff that the system call itself was doing。

And then we restore the user state which is going to include restoring the user's stack and then we return to user and the user picks up where they left off。

Okay。 And so yes I did say there's a separate kernel stack for every user process。

And there is a good comment in the in the chat does all of this security stuff rely on there being no bugs in the kernel。

Yes。 So if there are bugs in the kernel， all bets are off。 Some bugs are worse than others。

But you can imagine。 Okay。 Where is the kernel stack that's a great question can be many places in Pintas it turns out that the kernel stack actually is a single page and has the PCB in it。

Okay， not specifically in the part that's called a stack but it's all in one page。 Okay。

and we'll look。 And we'll look at that in more detail。

Why do we know the user can't can't touch the kernel stack because it'll be in a part of memory that's inaccessible because of the translation protection of the of the address。

base think back to base and bound from earlier。 These are in the gray part。

And as a result when we're running up here in green or yellow， they can't access gray。 Okay。

hopefully that helps。 And so we'll make sure that our protection is such that the user can't mess up the gray。

So for instance， in the case of the x86， you can see here that when we're running at the user level。

we have the the users stack。 We have the users code PC and what I'm showing you here is the stack segment。

And the ESP is a stack pointer here we have the code segment and the EIP。

And then after we're during， excuse me the interrupt system call。

we have switched atomically over to the kernel stack so notice how the stack pointer is on the exception or。

the back and the instruction pointer is pointing into kernel code。

This transfer from here to here actually happens as part of the hardware transfer in the x86 and we'll get a lot more detail about that but that's。

that's kind of how we set it up so that when the interrupt happens we go poof over the other side and we automatically switch to a new stack。

So that helps to keep the integrity of the kernel up there。 Okay。 Now， if you remember。

I gave you this picture last time but maybe we can understand a little bit more about it now so the application itself。

the user code is kind of up here at some， you know。

interesting program that is computing the last digit of pie or whatever。

And then we'll be linked with some standard libraries like libc that was mentioned in the chat earlier。

And then those libraries will potentially make system calls into a part of the kernel that is running part of the kernel code which is going to be running。

in the system mode。 And in the case of a monolithic kernel like this one。

we'll talk more about non monolithic ones much later。

All of that code is inside a special part of the memory that has been linked just for the kernel and set up to run in kernel mode and it has things like the file system and the scheduler and all sorts of things。

Okay， and we're going to talk about many different pieces of this as the term goes on。 And。

and this question in the， in the chat about why the registers are saved in the exception stack。

Think of the exception stack and the kernel stack are the same thing。 Okay， there， you know。

when you take an exception， the kernel stack is handling the exception。

So that's an exception stack at that point。 Okay， sorry for the confusion there。

But exception stack and kernel stack are the same for your purposes right now。

And so this system call interface is the one that we're going to spend a little bit more time on as we end this lecture and probably a little bit into the next lecture。

But I wanted to show you this right so all of the services that we think of as being provided securely by the kernel are things that run in kernel mode and are inside the kernels memory。

Now， there's a good question here is the hardware built with the OS in mind or is the OS built purely on top of the existing hardware。

So that's philosophy 436 that's a different class than this one。 I would say， you know。

originally there were no nothing that we would recognize as an OS but the OS idea has been around since almost the beginning of time and so basically。

Pretty much all hardware is built with an OS in mind。

It's just a question of how sophisticated an OS can run on given hardware and we'll hopefully understand a little bit more about that。

And then I love the chicken and egg comment in there。

The other reason there is a chicken and egg thing here is that as people have become more sophisticated in their requirements from the operating system。

hardware has adapted。 And the great example of that which we will talk about later in the term more is the idea of virtual machines。

There's been adaptation pretty significantly over the last 20 years of the way the CPU is designed to make it easier for running virtual machines。

Okay。 And the registers are the same registers。 So keep in mind， I'm going to say this。

The question here， do we have separated registers for the OS。

Usually no except for special processors。 For now， just remember。

we are thinking about one core or one CPU to rule them all。 Okay。

and it runs both the kernel code and the user code and we got to make sure the right thing happens with saving and restoring the registers when we're transitioning back and forth。

Okay， so think about a single， a single set of registers to rule them all is the way to think。 Okay。

so there's this narrow waste， which is really the system call interface and we're going to be exploring this more in detail。

but you could think of the space of applications is large。

the space of services and hardware is large， and it's the system call interface the。

controlled entry to the kernel that really gives us the power to make this work in a controlled way。

So what about that system call handler？ It's roughly similar to what interrupts。

but it's for calling services。 And so once again， we're going to vector through a standard。

well-defined system call entry points。 We're going to locate arguments and registers are on the stack or wherever when to get into the kernel。

We're going to copy the arguments because and make sure that they're validated and saying because we can't trust the user to either give us arguments that are in a reasonable part of memory or that are reasonable。

We're going to validate them and then we're going to copy the results back into user memory when we're done。

And so it's basically the kernel will trust， but verify in the sense that everything gets verified。

Maybe trust nothing， I guess。 But go into a system call， check everything， check it twice。

make sure all the addresses are good。 Do what you were asked to。

copy the results out of the kernel back into user space。

So the user is never given access to the kernel and it can't corrupt the kernel。

Now the question about does the， so I think the， let's hold off on the question of does the OS have its own registers or not。

I think keep in mind that most of the registers for execution are all the same。

There are some special ones for things like the base table of virtual memory and stuff。

but that's not kind of what I thought the question was about。 So， okay。

so let's put it all together。 Here's a web server， you know， you make a request to the web server。

you make a reply。

Question might be what is it that the web server does。

And so you can imagine on the web server itself is a process that is the web server。 Okay。

and on that web server， there's hardware with networking and disk。

There's the kernel and then there's the server process， which is that Apache web server。 All right。

and you know that web server when it starts up， it's a process。 It's running in user mode。

And the network， basically the server starts by saying read for the。

read the network socket that has been set up and ask it， are there more requests in。

And usually what will happen is that system called a read is going to wait by going to sleep if there's no requests。

And now in this situation， the server is sleeping because it's waiting for something。 Okay。

and meanwhile， the request， or sometime later the request will come in。

It'll generate an interrupt saying， hey， there's a request， which will wake up the server process。

return the request， and now the server process。 And now the server process has the request in a buffer。

So notice there was a read system call that came back with data， but in print。

what happened in timing is the read system call slept for a while until the interrupt came in。

And then we got our data。 And now that data is what it's a request for a web page。

So we figure out what that means。 It's HTTP and HTML。 And we're going to figure that out。

And now we know what file to read。 So we'll do a read system call。

Notice this read system call is going after the disk。 This read system call went after the network。

You're going to learn very soon that Unix makes everything that's I/O look pretty similar。

Reads and writes are going to be similar across the board。 But here we ask for a read。

That's going to have to wait because potentially the kernel may actually have to go to the disk。

And that's going to take time。 The disk will interrupt with a result。

The result will come back from the read。 Now we have a reply buffer。 We'll format the reply。

We'll send it back to the outgoing socket via another system call。

And the reply goes back and we end up with looking at a web page。 Okay。

And the difference between an interrupt and a system call here is the system call is a。

synchronous query from the process into the kernel that's like a function call that's。

going to give a result back。 Whereas the interrupt is an asynchronous thing that comes in from the outside world。

Okay。 All right。 And this step two and step seven， those are interrupts。 Okay。

And they're going to invoke that interrupt mechanism I showed you earlier。 Okay。

And the white box is here。 Uh， represent queuing。 And so we're， you know。

I'm being very high level here right now， but there's a bunch of， queues。

And so typically if there were a bunch of packets that came in before we did our read， then。

they'd be queued and the read would grab the first one and so on。

So typical sockets or buffers have many， uh， many， um， queues。 Now this question about， um。

does the interrupt happen without the kernel making a， request？ And the answer is going to be that。

um， yes， but there's some setup where we've set the， socket up in advance。 And once it's set up。

then there's a channel and somebody who's trying to make， uh， um。

get a web page will actually send a request in to a， to this queue that's been set up in， advance。

And so we're going to be able to make sure that we're not going to be able to make， the request。

And so we're going to be able to make sure that we're not going to be able to make， the request。

And so we're going to be able to make sure that we're not going to be able to make， the request。

And so we're going to be able to make sure that we're not going to be able to make， the request。

And so we're going to be able to make sure that we're not going to be able to make， the request。

And so we're going to be able to make sure that we're not going to be able to make， the request。

That's why， you know， we've got to make sure that when the interrupt happens， really。

this interrupt is actually interrupting something else that's running。

Probably not the service process because the service process is sleeping， but the。

interrupting something else that's happening， saving and restoring its registers， doing。

handling the interrupt and then restoring to that thing。

And then eventually we run the server process。 Okay。

And so there's a lot of simultaneous things going on here that are going to be things we。

have to sort out， but I am quite confident that within the next couple of weeks you'll。

have a much clean handle， a much better handle on how this all works together。

because there's a fairly simple set of mechanisms。

All right。 So if you recall and we don't have a time to go through this entirely， actually。

let me stop and see if there's one more question on this。 Anybody else have any other questions？ Ah。

good question。 What happens if an interrupt comes while another interrupt is being handled？

So remember that interrupt controller。 So the interrupt controller。

when the first interrupt comes in， okay， both， I'm assuming for the sake of argument。

you're asking what happens if two， interrupts come in from the network at the same time or roughly the same time。

The first one will enter the interrupt handler and that interrupt handler will。

disable the network interrupt while it's processing interrupt number one， taking。

that first packet in。 And then when it returns from interrupt， it'll re-enable and at that point。

the second interrupt will come and go forward。 Okay。

So basically the way we control the interrupts to make sure that we don't have。

chaos because the outside world is messy， right？ There's the philosophy 405 for you today that the outside world is messy。

We make sure that we never let， we never re-enable interrupts to happen if。

we're doing something to the core state of some thing inside the kernel that would。

get messed up if the second one came along。 And so that's how we're going to enable and disable interrupts properly。

Okay。 Good。 All right。 So if you remember， we talked about processes today， but one thing。

So how do we manage the process state？ You know， obviously you see we got stacks and registers and code and data。

We're going to have to figure out how to initialize all that stuff。

So we're going to get there and we're going to need to worry about creating， and exiting processes。

Okay。 And if you remember， everything outside of the kernel runs in a process。

So it's not like processes are special things in the sense that they only， happen occasionally。

Processes are major functionality。 Okay。 And so even more interesting is the fact that processes are created and。

managed by other processes。 So we have a bootstrapping problem here。

So processes are created by other processes。 You know， this is the ultimate chicken and the egg。

How does the first process start？ And the answer is the very first process gets started at the time of the。

kernels booted。 And it's typically called the init process。 And once that init process starts。

it will then start a shell potentially， or start a set of other processes。

So once we have the first process， it will then be able to do something， called fork， which will。

if you'll bear with me for a moment， I'll at least， give you an idea what it is。

And then we'll talk a lot more about it next time， which allows more。

processes to be created with that first one as the parent。 Okay。

And so we have a process management API exit fork， exact wait， kill， sig action。

And in your project number one， you're going to get to play with this， pretty intimately。

You'll get to learn this part of the API。 And for instance， what does exit do？ Well， exit。

as you can imagine， is a system call that takes us from running。

a user process to exiting the user process and basically destroying， its state。

And if you remember that little semi-circle picture I had where there。

was red in the middle and there was the blue user process， the thing at the， far right was exit。

So that's how a process is killed off。 And if you look here， you're going to learn a lot about main。

That's going to be your favorite procedure in C because that's the。

starting point for a typical process。 And we run a bunch of stuff here。 In this case。

we're running get PID， which is a libc function that， then makes a get PID system call。

So it asks the kernel， "Hey， what is my process？ ID？"， And we'll print it out with a printf。

And then we exit with a zero argument that typically means no errors。 So zero is a good exit。 Okay。

And the question might be， "What if we forgot that？"， And we just let main exit。 Well。

it turns out that main is not the real first thing that runs。 There's other stuff。

And so if we forget exit and we just let main and then exit gets。

called automatically for us by the OS library。 Okay。

So there's kind of an implicit exit zero at the end。 All right。 Okay。 Now you'll see this in。

you know， project zero and it。c。 You get to actually see kind of part of what happens at the。

beginning of wrapped around main there。 Okay。 Now let's look at fork。

So fork is one of the stranger routines that you'll get to run。

into part of its kind of a legacy thing from original Unix。

But it's actually a pretty useful routine。 So the idea is copy the current process and get a new process ID。

and create a brand new address space and a brand new security。

context and start running with a single thread。 Okay。

And that single thread is going to kind of pick up where the other one left， off。

So fork is the weirdest function call that you're going to get。

And that's the first function call that you're ever going to run。

into because if you think about it before you execute fork。

there's one process after you execute fork， there's two。

processes which are duplicates of each other。 So if you were to look inside。

what happens is the first process， executes fork and then it returns from fork。

And the second one also returns from fork in the exact same place。

in the code except that the original one， which is the parent， process。

returns something non-zero from fork。 It's an integer and that non-zero number is actually the process。

ID of the child。 Okay。 And in the child process， it's going to return zero telling that。

guy that it's child。 And if there's something less than zero， that's going to be an error。

And that's only going to occur inside the parent。 Okay。

And the state of the original process is duplicated in both， the parent and the child。 Okay。

There's going to be duplicate address spaces。 So the address space is going to be fully copied into the child。

All of the file descriptors are going to be copied。 Everything is going to be copied。

And you'll have two things that are running and are essentially， identical for each other。

But one of them gets a non-zero back from fork and the other gets a， zero。 Okay。

Now the use case is going to be something you're going to run， into very quickly。

which is how a shell works。 It uses fork among other things。 Okay。

And the child is able to find out who its parent is and what its。

parent ID is by a separate system call。 So you'll learn about those。

But the other thing to think about a use case for fork is suppose。

that you have somebody running and you fork off a child process。

and it's going to have exactly the same environment as a parent。

So it'll be easy for you to figure out how to make it。 And at that point。

the child will be in that environment you've， already set up and you can have the parent wait until it's done。

Okay。 And if you look， I'm going to give you one quick example here。

How does it decide greater than zero versus zero？ Well， that's just an if clause。 So here you say。

you know， result equal fork and then if result， greater than zero， otherwise if result zero。

And those of you who can bear with me for another couple of minutes。

I want to just show you an example。 So here's a good example。

So here's a fork usage where we have the main。 Okay。 And it sort of gets its own PID and it says。

oh， parent PID is this。 All right。 And now at this point， we call fork and we set the CPID to equal。

the return from fork。 Okay。 And notice what we do right after。 We say if CPID greater than zero。

then we know that the parent is， going to run in this part of the if else if CPID is zero。

the child is going to run in this part of fork。 Otherwise we're going to cause an error or declare an error and say。

the fork failed。 Okay。 Yes。 This is messed up。 Maybe。 Well， it's a good thing to learn。 Okay。

Now notice that first and foremost， after you get past fork。

there are two things that are essentially identical where the only。

difference is one of the processes CPID is greater than zero and the。

other process CPID is equal to zero。 And the question of what do you。

what if you call a function inside， either of these if statements？ Well。

that function will now run only in the given process。 So up to this point， we had only one process。

After this point， we have two。 If you run something in this arm of the if， it'll only run in the。

parent process。 If you run something in this arm of the if， it'll only run in the， child process。

So for instance， here I've got two arrows。 So I've got C and P。

You can imagine C stands for parent and P stands， for child。 Oh wait， are you paying attention？

C stands for child and P stands for parent。 And notice that right after fork。

we have two things and they'll， start executing。 And in that case。

the parent will run here and the child will run， here。 Okay。

When does fork return one and when does it return the other？ It always returns。

unless there's a failure， it returns greater than。

zero in one process and zero in the other process。

So this is why I'm leaving you guys to mull over this over dinner， to think about this。

After we call fork， there are now two where there were just one。

Those two processes are both running。 One of them， the return from fork was greater than zero。

The other one， the return from fork was zero。 Okay。 Now the question of what's the PID of the child。

So every process has a process ID。 Here we know what the child's PID was just by looking at CPID。

Here the child has to ask， you know， get PID to get its own PID。 Okay。

Now there's lots of questions here。 One of them， why do we have to explicitly call fork？ Well。

because fork is a style of parallelism。 So fork is one way to do it。

We'll talk about other styles of parallelism as we go。

But fork is the principle way that you have for creating a brand new， protection environment。

namely a process。 Okay。 We'll talk about new threads。 So this new process that we've just created。

the child process has， exactly one thread running。 We'll talk about how to get more。 Okay。

And the child process can yes call fork。 So in here， the child process could go ahead and call fork。

And now we've got a parent， a child and a grandchild。 Okay。 Or the child could call fork and a loop。

And now you have a parent， a child and a bunch of grandchildren。 Okay。

So there are many ways to making， uh， yeah， while one fork。 Okay。

So that's a guaranteed crash of your machine。 Okay。 Um。

actually what will happen at some point is you'll probably exceed some， uh。

number of processes that you're allowed。 Okay。 All right。 Um， and there is a limit。 Uh， okay。 So。

um， we're gonna， we're gonna end there， but I just wanted you to see。 So fork。

think of this as one way to get parallelism。 Okay。 And we'll talk a lot more about fork， uh。

next time and keep in mind that， there'll be many other ways to create parallelism。 All right。

So in conclusion， we talked a lot about processes this time。 Okay。

Address based with one or more threads is a process。 Uh， it owns the process owns its address space。

Inside that address space， what happens when you read or write the address 12，468 could。

vary from every process。 Okay。 We'll get to that later。 The process owns its own file descriptors。

file system context and has one or more， threads。 We talked about how interrupts are hardware mechanism for gaining control from。

outside to run in the kernel。 And it's a notification that events have happened where the big one is going to。

be timers for us， but also IO like network and， and kernel or a network and disk。

And we also talked， started talking about native control of processes。 We talked about fork， uh。

and we talked about exit and we'll get into exact and so on。 Uh， next time。

so I'm going to leave you go because we're way over time， but I hope， you， uh。

guys have a great evening and we'll see you later。 Bye。 Yeah。

[BLANK_AUDIO]。

P4：Lecture 4： Fork (Con't), Introduction to I O (Everything is a File!) - RubatoTheEmber - BV1L541117gr

Okay， welcome back everybody to 162。

We're going to pick up where we left off last time briefly and finish our discussion of。

fork and then move on。 And so if you remember last time we were talking about how to create new processes。

And basically the mechanism that we talked about was fork。

This is going to be one of several mechanisms for getting parallelism and concurrency that we're going to discuss over this term。

And if you remember fork was a call that was function call that was a little weird relative to anything else you've ever seen before。

So what fork does is it you call fork in one process and it returns in two different ones。 Okay。

so you have a parent process you call fork。 It creates a child process。

which is identical to the parent one and then both of them return from the fork system call。

And if you notice， one of them will return something bigger than zero and the other will return something equal to zero。

If it's bigger than zero， that's going to be the original parent process。

And the number you get back is the process ID for the child。

If you get zero then the process knows that it was the child process that it's in and it can call get PID and get parent PID and all of those to find out kind of what's parent is and what。

it's ideas。 Only if you get something less than zero then you know that the fork process failed and you're still running in the parent and the number that you get back tells you something about an error code。

Okay， and the key thing about fork is that the state of the original process ends up duplicated in both the parent and the child。

So if there are gigabytes of memory。 In one case in the parent when you fork the child will have gigabytes of memory。

Okay， and that may seem kind of excessive except that what's going to happen is we're going to show you how that's actually not too expensive because really we're not going to copy all of the memory we're just going to copy the page tables and in the parent and the child so that we can sort of。

duplicate the memory without actually having to duplicate anything。

That'll make for another discussion。 So， so remember two processes are there after fork and both of them return from fork。

And so what I'm showing you here is what happens right after we call fork。

the value that comes back is an integer。 Okay， it's a PID T actually but it's that's really an integer and both the child and the parent process start from that same spot。

If， and then they start running。 And what you'll see is that the parent will notice that CP IDs greater than zero start running in this point。

The child will notice that CP ID is equal to zero start running in this point。

And at that from that point on then that the executions of the two processes diverge。

And you can do something completely different in the parent than from the child。 Okay。

now the question in this chat there does this， you know， the parent and the child share the memory。

So the answer is yes and no。 And so I don't want to get too much in the subtleties you should think of this as if they get complete copies。

The way this is really going to work is we're going to set everything read only。

And the moment one of them tries to write then we'll copy and so it'll still look exactly like they have their own separate copies。

but it's a lot cheaper than copying all the memory。 Okay。

we'll talk more about that in a couple of lectures。 So。

I want to see if there are any questions on this basic idea here。

and then we're going to go a little forward on it。 Okay。

to one process calls for two of them returned from four。 All right， now。

here's a good challenge for you guys。 So here is a program。

Notice what it does is it has an integer that's declared that's a global variable。

It calls for the parent will run in one arm of the L of the F the child or run in the other and I'm ignoring errors here for a moment。

Question is， what do we see on the screen if this runs because it looks like I the two eyes uses of I are kind of interfering with each other when I'm going up one of them's going down the two processes are running at same time potentially what happens。

Very good。 So somebody said in chat wait， isn't it the fact the processes are protected from each other。

Yes。 Good answer。 So if you notice， yes， the parent had a global variable I。

And when we duplicate they both have a global variable I but it's a different global variable because it's a completely different address space。

So in fact， these two processes will do their loops perfectly happily with I going up and one and down in the other。

The thing that we'll see that's going to be a little non deterministic is since they're both printing to the same standard out and we'll get to that later in the lecture。

then you'll see some interleaving from the two of them but there won't be any confusion about I。

Okay。 So， so basically I is is literally the execution graph is forking into two completely unrelated things。

Yes。 That was to we talked about exit last time that was for。

I want to give you a couple more sort of process management pieces here before we before we move on to the next topic。

So what I've shown you so far gives you new processes but they're kind of duplicates of the each other and that hasn't seen too useful yet if you want to make a bunch of new processes。

Okay。 To do that we're going to have the exact system call and wait is going to help parents understand about children finishing execution。

So let's look at the exact first。 So the exact system call and there's a bunch of different variants of that so I'm going to call them exact generically。

You can do a man on exec to find out all the different versions。

but what's going to happen here is if you notice we start off with fork。 When we're in the parent。

all we're going to do is we're going to execute weight which is going to just wait until the child exits and I'll say more about that in the next slide。

And the child is going to do something interesting it's going to make an array of of character strings and notice this is LS dash L and then a no。

So it's an array with two strings and a no。 And the exact V system call is going to have its first argument slash bin slash LS and the second argument is going to be this array of arguments。

Okay， and what's interesting about this is we for to create two processes the parent just spin weights。

It's actually just waiting for a moment。 The child doesn't exec。

And what that does is it looks this program up on the file system and it replaces everything that was in the process with the contents of the new program and it starts at running。

Okay， so assuming there are no errors。 What we just did here is we forked off a new process。

a child process。 We started it running the LS program。 And the parent is waiting。 Okay。

and so now you can start to see how like your shell might work。

where the shell has its parent which has the little prompt that puts down there and processes what you type。

And all of the things you spawn off to run get put in their own process as child processes。 Okay。

and so what exec does is assuming there's no failure。

Exact will throw out all the memory that's in an existing process and it'll start a new program running in that process。

So the process sticks around the contents of the address space of that process changes。

And the only reason that you would ever return from exec is if there was an error。 Okay。

And then what about this weight system call so there are lots of versions of weight you can look that up。

The simplest one that I told you about here is trying to address an interesting problem you can imagine。

So if the parent starts a child process and the child process ends。

the parent wants to know what was the success of that process。

Was there a successful return was there an error， etc。 And so weight basically lets us do that。

And so if you notice down here the child is basically executing something。

And then it doesn't exit 42 which since that's not zero is technically an error。

The parent does a weight and it gives the address of an integer variable for a return value。

And this weight system call will wait until the child exits at which point this exit will get the exit code will get put into the variable that we've put forward。

And the weight system call will return and we can see what the return value was from the child。

Okay。 Yeah， and so the， and the question about back here whoops is that yes the ARGS are basically just like the arguments to LS。

And that's essentially how the arguments to LS get put in there。 Okay。 Now。

so again this weight system call waits for the child to exit gets its value and then moves forward。

Okay。 And so the reason you can think that this needs to be more complicated。

And so the reason why it's not clear is because if a child exits a long time before the parents ready to execute weight。

clearly that process has to sit around with its return code， even though it's done。

so that the parent can eventually get the weight status。 Okay。 And so if that's the case。

then that process that was a child， but doesn't have a isn't running anymore is what's called a zombie process and it'll sit around until somebody executes weight。

Now there's a good question about what happens if you have multiple children that are spawned off this weight system call this version will wait for the next one to exit。

There's a bunch of other ones where you can even say wait for this particular process ID to exit。

etc。 So there's much more sophisticated ones that you can do。

The other question here is does the exec the process here have access to all the parents memory。

And the answer is no。 So what happens is there's a child's memory。 Okay。

which is a copy but separate from the parents and exec will essentially overwrite everything with the new execution image。

Okay。 And so here's a， here's a typical what I call the shell pattern。

So this is what you typically see in your shell。 After you've typed your command。

we fork off a new process。 The child goes off and exec executes that process with an exec。

The parent does a wait and waits for it to finish。 Okay。

And that's what you typically see when you type a command on the shell you hit return something happens and then it gives you back to the shell。

That would be this。 Now， in a different pattern where you want to start a child and then go on。

let it run and go on to do something else in the shell。

That's going to be a case where we don't do wait， but we just move on。 Okay。 So the question is。

where is the status variable coming from if you look in this particular code。

notice it's the exit code。 So when the child exits with exit 42。

that is the status code that gets returned into the status variable。 Okay。 All right。 And then last。

I want to talk about。 So we were talking about starting， stopping， managing processes。

The last thing， that's from the standpoint kind of of a parent。

The system needs to be able to control processes that are already running。

And so that's basically the so called kill。 Action。 Okay。 And basically。

I'm going to show you an example here。 So typically， if you hit like control C。

That's actually going to send a signal that's going to be an interrupt signal to the process that's running。

And you want to， we have a more general idea here where we can send signals to any process。

And it's kind of like a user level version of an interrupt。

And you're going to actually get to play with this in your projects。

But I want to just show you a particularly simple example of what you can do。

So when you hit control C， the default thing is that the system will catch that control C and throw it off and kill it。

Okay。 However， if you want to do something different， you can do this。

So notice that the main procedure actually sets up a SIG action， which you can look at this code。

I don't want to go in great detail right now。 But the SIG， SIG action has a handler。

which is a program， which is a function that you've actually registered already。

So here's the signal callback handler that you've put in your code。

you register that as a handler to be called on SIG it。 And so when you hit control C。

it'll now call this instead of the default systems version。 And in this case。

it'll say caught signal and then die。 But you could have it say caught signal ignoring and return immediately。

And that means the control C wouldn't do anything except keep saying caught signal。

So there's lots of flexibility there with signals。 Okay。 All right。

And then there's a bunch of different signals that SIG and SIG into control C。 There's a SIG term。

which is the kill shell command。 And then there's like SIG kill and SIG stop。

which are uncatcheable signals。 SIG kill is what you get with kill minus 15 for kill minus nine。

You get these extreme signals that are not catchable。 Okay。

So I guess I'm not entirely understanding the question here why just the status variable。

This is a code and the selected variable so status in this case。 So status。

the word status here is just because I said status notice that my city and status this could be a variable of any name。

So this could be weight。 Fred， if you wanted， I don't know if that answers your question。

whatever you， whatever you feel like calling that variable in the parent you can do it。 Okay。

All right， so moving on now if you remember last time and the time before actually we showed you this figure。

which kind of gave you an idea of the difference between user mode and kernel mode。

So the kernel mode is the set of high priority things that are inside the kernel and they have direct access to the hardware and they have to be perfect。

Okay。 The user mode are the things that are written by users and other people and may have bugs in them and typically what the kernel is doing is it's providing the process of traction and making sure that the protections of the file system。

and all of those other things are maintained by the operating system。

All of that's inside the kernel。 The user mode is the thing that is running most of your programs。

but is running inside a process address space and protected from all the other ones and all the other ones are protected from it。

Okay。 So， the， the difference between the kernel and the shell is a question the shell is just a program running in user mode that you're running on your own。

but it's asking the kernel to start up new process。

So the kernel is this core bit of the operating system that handles the file system and the scheduling and all of that sort of stuff and is essentially always there。

Okay， the shell is just a user mode program like every other one it's just it's been set up to process your commands and use those to launch programs running。

Okay， and again if there with respect to wait if there's multiple children then wait we'll just wait for the next one to exit。

And then you can run it again and again。 Okay， so the other thing to note by the way is this there's two things shown in user mode here kind of the applications in the standard libraries。

So the standard libraries are a set of wrappers around the system call interface to make it somewhat easier to use and just the raw system call interface。

And so if you look here at this idea of kind of the narrow waste of the system call interface puts the library like a lib C is a good example of that on top of the system call interface。

and the things inside a lib C are somewhat easier to use。

than just the raw system calls。 And so most people who are programming in see at least link with lib C and so they don't use system calls directly what they do is they use lib C which uses system calls。

Okay。 And so， for instance， I had this picture earlier and I just wanted to clarify so typically you have a library like a library。

and you can see that there's a library。

And to clarify so typically you have a library like lib C that is linked into your application this is showing a bunch of different applications grew green yellow orange with the lib C and the lib C gives a nice clean easy interface up to the application。

And it calls the OS typically， and most of the library code runs inside of the user mode with occasional call outs to system calls inside the OS。

Yeah， as Anthony says about weight I know that's a great interest to a bunch of people but do man wait and you'll see there are lots of different versions of it。

Many that do， you know， non blocking things that let you say， well wait for this particular process。

etc。

Okay。 So， this idea here of wrapping things around the system call interface is a very standard thing because that system call interfaces extremely powerful because it's like a function call into the kernel and out。

but it's very stripped down it's kind of the bare minimum that you need。

Okay。 And we're going to today today we're going to talk a lot about going across that system call interface。

And I'm going to give you some good examples of where lib C comes into play to provide a cleaner wrapper around the actual calls into the system call interface。

Okay， and this should help set you up quite well for project number one。

And so the first idea for the day here is that everything is a file。 Okay。

and this is a very you Unix idea。 And it got adopted and standardized as posits and I'll say what posits is a moment but it's a standardization of a lot of Unix。

And the idea behind this is it is an identical interface for files devices， regular files on disk。

networking， interprocess communication， all of those， everything looks like a file。 Okay。

And so a lot of this type of communication， things that go across that system call interface are based on system calls like open read right close。

which you're going to become very familiar with by the end of the term。 Okay。

and there's an additional part of this interface which we haven't talked about yet， called I octals。

and the I octals are kind of or I could all I've heard people say。

although that's a little strange sounding to me。 But the I octal interface lets you do customize stuff to interfaces when you can't quite shoehorn everything into open read writer close。

And this typically an I octal interface that lets you do some additional configuration to those things and we'll talk about a few I octal calls over the term。

but basically imagine that open read right and close are what you do for everything， almost。

And the idea that everything is a file was actually kind of radical when it was originally proposed。

and you can see this in the one of their early papers from 1974 from Richie and Thompson called the Unix time sharing system。

And I actually posted this on the resources page， if you're curious about it it's one of its actually the first paper we read in 262 a so you can get a little bit of of an interesting flavor for what a graduate computer systems it's like。

And this idea of everything's a file was powerful enough that now we don't even think of it as radical。

and it even got adopted as a standardized interface so I wanted to say what POSIX is so POSIX is the so called portable operating system interface for Unix。

It's an interface for application programmers， specifically it's a system called interface。

and it was created to kind of bring order to the fact that there were many different flavors of Unix spread all over the place and they all had slightly different system。

call interfaces。 And so what happened was they adopted a more standardized version called POSIX。

and some of the POSIX interfaces are even available in Windows。

even though Windows is not a Unix operating system that still has POSIX interfaces。

And so when I talk about Unix system calls and so on。

mostly I'm talking about things that are standardized in POSIX and if I'm not， I will。

I will clarify。 So let's look at this ever present file system abstraction。

So a file is a named collection of data in a file system， for instance。

So POSIX file data is a sequence of bytes could be text or binary or serialized objects it doesn't really matter。

and the operating system doesn't care so the operating system doesn't have any interest in how you format your data it's just going to give you kind of a bag of。

bytes that are sequenced， and it's up to you to interpret them。 Okay。

and there's only one example of a file type that's actually processed directly by the Unix file system and that's directories are in a special format。

And we'll get to that later in the term but for now a file is a sequence of bytes。

It doesn't matter what the bytes represent。 Okay， and then there's file metadata like size。

modification time， owner， security information， all of that stuff， which。

you'll you'll get a good flavor for as you start using this interface a lot more。

A directory is really just another special file containing pointers to files and other directories with names。

Okay。 And so each directory sort of provides a mapping between the file name and。

either a directory or a file that's represented there。

And if you trace your way from the root directory， which is the very top level。

through a set of directories to a file you're actually hopping your way through the。

route directory， which is a file to the next directory， which is a file to the next directory。

which is a file to the final file， which is not a directory。

That tracing of the path is something that we're going to talk a lot about when we get into file systems in several weeks。

Okay。 But you've all used this so you're familiar with this idea。 And inside of Unix。

every process actually has its own current working directory。

which is kind of the directory path for files that are being used by that process。

So there's a system call change directory， that can you be used to set the current working directory。

And absolute paths， which are things that start with a slash kind of ignore the current working directory。

So this one is at root file system slash home slash osuki slash six。

CS 6162 or relative paths are things with dots in them。 Okay。

or in some cases with tilde's those are relative to the current working directory。

So if you just say index， HTML， that's really current working directory slash index dot HTML。 Okay。

Now， what about this interface to files and so what I want to do today is we're going to do a really quick tour of the levels of the stack for IO starting with the high level of streams。

which is buffered IO and that's in libc down through the low level， through the Cisco interface。

We're not going to say a lot about file systems right now but we're also going to say a little bit about IO device drivers。

just so you can get a flavor of all these interesting pieces of the stack that we're going to cover as the term goes on。

So at the highest level。 Okay， we have what are called streams。 Okay。

and the C high level API operates on stream。 So stream is an as a sequence of bytes that isn't really formatted in any particular way。

And it's got a position associated with it。 I've got this little arrow here。

And there is an F open and F close routine。 These are a lot of these are in libc to open a file or close a file。

And notice that there's a special F in front of here F open F close。 So these aren't open and close。

So these are the F versions or the stream versions。 And so F open takes a file name。

which could either be an absolute path， or something relative to current working directory。

And then some mode bits。 And it returns something called a file star which is a is a structure that you don't look into。

but is a stream that you can read and write from。 And then when you're done you F close you close by giving it back the file star。

Now remember what this means is there is some structure of a file capital file and it returns a pointer to that。

And then the definitions are for this standard I/O dot H include all the definitions you need for this。

Okay， now the question here about what's a stream I'm going to say more in a moment okay but it's really just like I said it's an。

unformatted sequence of bytes that are kind of streaming along and you're going to read them。

Just read a whole sequence of bytes okay it's going to be a stream of bytes。

And it's going to be like a river。 Okay， these mode bits are really something that you give it a string。

a pointer to a set of characters。 And there's a lot of mode possibilities here so if you give it an R。

it'll be read only if you give it a W it's right only if you give it an A you're opening it for a pending。

which is starting at the end， et cetera。 Now， now the question about are the permissions stored in metadata for files。

So， I'm going to say yes for things that are of the power of Unix。

We're going to actually look at the fat file system as our first one where permissions are a little bit less cleanly defined but for now it's actually。

part of the metadata。 Okay。 And so an open stream is actually represented by a pointer to a file structure so look what comes back from F open is a file star。

Okay。 And how do you get an error back so this is going to be an essential question that I'm going to want to train you guys and Anthony's going to want to train you guys very well by the end of the term。

Whenever you execute a system function of some sort you always ask yourself what's the error condition。

And here the error condition would be that rather than getting a pointer to a file structure you get back a null which are zero。

Okay， in which case the F open failed and you shouldn't go anywhere with it。 Okay。

And in class I'll say more about this later， I will probably not always in fact frequently won't look at the error cases because that would just make my slides messy。

but you as budding system programmers should always think about errors。 And what's the error return。

And let's look at some of what we got with the stream interface so among other things。

there are some free standard streams that are always open for reading and writing。

So that's the standard input standard output and standard error。

So standard in if you read from it that's the normal source of input。 Okay。

which if you don't do anything else is typically like the keyboard。 Okay。

standard out is an output stream and if you don't do anything special will just print on your screen。

Standard error also prints on your screen but typically errors go to standard error。

So standard out in standard error， unless you change them are interleaved， typically。

and they go on the same screen。 Okay。 Unlike a regular file where I have to execute F open to get this in every process。

you automatically have standard in standard out and standard error ready to go。 Okay。 And basically。

they enable composition and Unix in an interesting way。

So one thing you might not have done yet but you should in your in your Unix environment is if you say like cat。

which means dump out the contents of hello。text and you put this little， vertical bar。

which is a pipe symbol and then you say grep world。

What you're really doing is you're taking the output of this process。

The shell will start two processes。 The output of the first one goes to standard out。

The input of the second one for looking for matches comes from standard in but the little pipe symbol actually connects together the standard out of this process。

to the standard in of this process automatically。 And as a result。

what this will do is it'll take all the contents of hello。

text and send it to grep to look for the world world， the word world exclamation point in that file。

Okay。 And so we're actually connecting up standard ins and standard outs with the shell and that's going to be something that you get to do with the shell homework。

Okay。 So that's something to look forward to。 Now。 Let's look。

So the good question here is in the chat if it was just grep what would happen。

So the answer is that really the way you would do this is you would say grep world and then you'd say the file name and so then grep actually has two arguments that you use。

Okay， if you just said grep quote world on the line by itself。

it would pause there because it'd be waiting for standard input。

So you would actually end up with a with a failure in that case okay because you really need to have a file to to grep from。

Okay， but get that a try。 Okay。 And it's actually what will really happen。 Well。

you give it a try see what happens。 Okay。 So now we're going to。

So there was another question here about what does the greater than symbol do we're not we're not going to get too in depth into the into the details here until you get a little bit later but if you if I were to put a greater than symbol。

After all of this and then put a file name what it says is take the standard out from that grep process and dump it into a file。

So the greater than symbol， rather than the bar actually allows you to redirect into a file。

So part of what the shell homework is going to be is you get to learn how to make vertical bar and greater than and less than symbols work properly by tying together processes。

So that's actually what you get to do。 So let's look at something that's not standard it in standard out so if they let's suppose that we had a file on disk and we wanted to do stuff with it well once we opened it。

Then we've got the we can put the file star for that file we've opened here and now we can do reads and rights of single characters or strains。

Okay。 And this will return this will put F put C will try to put the value on to a stream so this is a right。

And it'll either return the character or an error。

Here's a read for instance of string will actually read up till or this will excuse me F put as this will actually put a string onto the file all the way up to the no。

And then there's other ones like this okay so these are character oriented getting okay。

And you can do man so I'm going to want you to start looking at man。

And then giving a command and that'll actually give you exact details about what these do。 Okay。

another version so but let's look at it one example just you see so notice how I opened input。

text this is all relative to the current working directory。

I get that input file star I have an output file output file star notice that the input one is open for reading the output ones open for writing。

And then what I do is I get a character from from the input file。

And as long as that character is not EOF I write it to the output file I get the next one and I keep in the loop so that'll basically transfer everything from the input file to the output file。

And then I close the two okay and this is doing character by character IO。 Okay。

so in that previous example by the way the question is was cat and grapped two different children processes at the same time yes。

Okay so now those were single character or bite oriented IO we also have block oriented IO again notice they all have F in front of them that's important。

So in addition to the file star of the open file we in this case we actually give a pointer to a buffer and the size of elements that we want and how many of them we want to read or write。

And this will read a whole bunch of things at once。 Okay， and that will be that's buffered。

Bigger IO okay and it into a buffer that you specified and so this void star is really just a pointer to a buffer that you've pre allocated of an arbitrary type that's kind of what void star means and see you get used to this as well。

And so with F read I can read a big chunk of data at once rather than a single character and with F right I can write a big chunk of things rather than a single character at a time。

And you can imagine this has a lot lower overhead and so you'd want to do buffer reader right you can't。

Okay， and so here's another example of using the buffer read and write versions and notice what I've done here or the read F right and F read versions and notice I define my buffer size here as a constant for 1024。

And down inside the main here I declare a buffer of size 1024 that's what this means it's a character buffer。

And so then what I'm going to do is I read into that buffer。

That's a pointer to buffer of maximum size buffer size， a bunch of elements of size。

whatever character is。 Okay， and the， and from the input file。

And that comes back as a link that tells us what length was come back with and it's either non zero or zero if it's zero the files done if it's non zero。

then I go ahead and write the buffer out that many items and I keep looping。

So this is now doing a much better copy from input that text to output。

that text where we potentially grabbing things in 1024 by chunks it's much more efficient。 Okay， so。

here， we're still doing everything is still a stream but we're doing block oriented IO here and it was character oriented IO in the previous one so we're all still streaming。

Alright， and again check your errors so always assistant programmers。

you should be really writing things like this where we get the input from F open， if the inputs， no。

we do something else because we failed you do not want to pass no。

You want to pass no into one like F read if this were no then this thing would fail in a bad way okay and you wouldn't necessarily know why。

Alright， always check errors， always start by saying man。

give the command see what the error return is like。 That's not error checking in class。

but that doesn't mean that you shouldn't check that。 Alright， so that's the high level API。

There are a bunch of other things like you can do F seek to set the pointer to a particular part in the file。

Okay， so F seek you can set it relative to the， the current position so here's the current position of the file。

you can seek to an offset from there you can seek to an offset from the beginning。

And thereby you can go to different parts of the file to read different parts randomly as you will。

Okay， but if you don't seek， then each read， whether it's a。

whether it's a character oriented get see or a block oriented read will actually just work its way slowly because it advances the pointer as you go。

Okay， so now what I've just described with streams is actually a good example of Lib C wrapping an interface around the low level descriptors for you。

Okay， and the reason that this is helpful。 A particularly good reason is if we go back to this example of getting one character at a time and writing one character at a time。

you can imagine this is horribly inefficient。 In general。

because what happens is if you made a system call for every character that you were getting。

you got to go into the kernel that's got a lot of overhead it's got to go find the character bring it back。

And so this would be her horrendously inefficient。

except we're using the streaming versions because we've got F here。

And what happens is F get see is actually running it user level inside the kernel， excuse me。

running it user level inside the process。 And this file descriptor actually has a chunk of memory that it has reserved in user level for for buffering。

So when you do F get see the first time it actually goes into the kernel grabs a chunk of data like a thousand 24 bytes and puts it into the user level buffer。

And then each F gets see from that point on until we run out of that data is really efficient because it's in user level。

So the streaming is really about automatic buffering in a way in live see without you having to worry about it。

Okay。 So， but that's implemented at the low level by actually using the real system calls here's open create close and this is not a misspelling。

of create by the way， these are system calls directly that go into the kernel。

And here I give a file named open and some flags in a mode， for instance。

and it opens the file for me but it returns an integer not a file star that means it's returning a file descriptor。

Okay。 And that file descriptor integer is something that I'm then going to use from that point on when I'm reading and writing。

And I'll show you that in a moment。 Okay。 But so what to notice from this。

There are some flags like read， write， create， etc。

And more bits like permissions that I need for when I'm writing， you know。

what's the user group other permissions that I want to put in that file。

Notice that return from open， assuming that it's not an error which would be negative return。

What I get back is an integer file descriptor that is an integer。 Okay。

And the other thing I want to point out is this open doesn't have an F in front of it。

It's open without an F。 Okay。 And so if the air if what comes back from opens negative it's an error。

Otherwise it's a file descriptor。 Okay。 And just like， just like in the buffered stream versions。

the， the un buffered low level versions also have file descriptors that are integers for standard in standard out on standard error。

But since they're integers， what we know is there zero is standard in one is standard out and to a standard error。

And every process typically starts with those values assigned to something。 And so standard in。

which is， which is zero is assigned to grab stuff from your keyboard and standard out and standard error typically set up to send things to the screen。

Okay。 And there are macros that you want to use。 You don't want to say 012 because that's bad programming style。

You want to say standard in file， no， etc。 And these macros are defined in another include file。

you understand。h。 Okay。 All right。 Now， so the difference between the F buffered versions and the un buffered versions is the un buffered versions don't have that buffer in memory。

Okay。 So they're， they don't have that efficiency of asking the kernel for more than the user currently wanted so that they can be more efficient。

Okay。 But this is the direct interface into the kernel。 So， so notice。

for instance that read now doesn't have an F in front of it because I'm using the low level interface。

I put the file descriptor as the first argument。 And then I have a buffer and a size。

which is the maximum number of characters that I can get back。 Okay。

And that's a read and reads will read up to the maximum size I asked for， but it might be less。

Okay。 And the， um， notice the fact here that what comes back from read is one of two possibilities。

either it's a zero， in which case， not only was nothing read。

but the file is done where at the end of file。 Otherwise。

it's some number greater than one to tell me how many characters I got。 Okay。

And it actually might be less than what I asked for。 So if I asked for 20。

it might give me back one。 And you have to keep that in mind。 Okay。

So it returns up to maximum size。 And so if you're expecting more。

you may have to do this in a loop。 Okay。 And as is usual， if it's negative， it's an error。 Okay。

And writing has a similar idea。 Notice the file is crypto is the first argument。 It's an integer。

I have the buffer and how much I want to write， and it returns number of bytes written。

And in this case， if the only reason it would return less than the total number of bytes I asked for is typically an error。

but you should do man on， right as well。 Okay。 And the reason that a file descriptor is an integer。

Well， because it's going to be a look up in a table inside the kernel。 So everybody。

I'm going to have you back up for a moment and think about what we said about interrupt handlers。

right？ The interrupt handler had a vector， and we took the interrupt number and that told us a very clean entry point into the kernel。

And so that was one way of being secure against the user doing something crazy。

This example where the file descriptor is just an integer is the same idea where it gives me back an integer and then I pass a data integer to the kernel when I'm busy reading and writing。

And the kernel， the kernel is not trusting a file star， which is a buffer and。

a user space that might be bogus。 It's only got an integer and it can directly check and see whether that integer is good or not。

Okay。 But anyway， so read， write， seek。 Okay。 All right。

these are all versions of what we just saw in a buffering sense。 Okay。

And so here's a simple example of I open a file for reading。 Okay。 And。

and then I have some permission bits here。 And I read it。 Okay。

I close it and then I write it to standard out。 So this is a very simple example。 Okay。

How many bytes does this program read well it's going to read up to 1000 right。

And then the difference a file descriptor which you see right here is the thing that comes back from open that is a file descriptor。

Okay。 It's an integer zero。 It's a positive integer。

If you ever get back a negative here you know there's a problem。 Okay。 So。

I want to emphasize something here， which is this design pattern that we've been talking about both at the low and the high level。

the low level design pattern is you always open before use。 Okay。

so notice we always did open so open is the thing or F open if you're using the streaming versions。

And then the thing that checks permissions and make sure you have permission to use something and assuming open returns without error。

then， then you no longer have to check the permissions because now you have a file descriptor that you can keep using for reading and writing。

And assuming that you use it the way you said you wanted to saying you wanted to read or wanted to write it should work。

Okay， and so we do all the permissions checking first and then all the subsequent reads and rights don't do permission checking。

So if you look here for instance， we did the permission checking at this point。

And then now when we do read we only give the file descriptor and we don't have to say anymore about what's the file name or any of that stuff because open now has a working file descriptor and we just。

do not。 Okay， and it's an index the file descriptors and index into internal tables and we'll talk a lot more about that as we go further。

Okay。 And so the pattern here is we always open first。

It's bite oriented so it doesn't matter whether I grab a big chunk of of bites or not。

I'm guessing it is bite so I sort of say I want a thousand 24 bites， or I want one bite。 Okay。

it doesn't care。 Oh， it doesn't care what it is or what those bites represent it's just giving me bites back or taking bites for rights。

We close it explicitly。 Okay。 So， reads inside the kernel are actually buffered。

So because a disk as you'll learn a lot later in the term takes things in thousand 24 or four thousand ninety six bite chunks。

It doesn't even make sense to try to read a bite from a disk。

So in the low level interface where you you're using the system calls and you asked for a couple of bites。

The kernel actually has its own cache， which we'll talk about， about。 So。

we'll talk about the buffer cache， which is storing chunks of things off of disk and feeding them to the user in a way that lets it look like everything is bite oriented。

So that's the everything's a file。 Everything's a bite oriented file and it doesn't matter that the underlying storage is block based we give that illusion of everything's a file and it's all bite oriented。

So， I'm not sure if we're in the kernel because you can't write a single bite to a disk either。

Okay。 And so all of that stuff is buffered inside of the kernel。

And this buffering is sort of part of global buffer management across the whole machine。

And to make things complicated perhaps is that buffering is done by the operating system。

The stream buffering is done by the user level library and those two buffers could cause you trouble if you weren't careful and remembering。

what's buffered where。 Okay。 But you'll get used to that。 Don't worry。

And we'll make sure that makes sense in addition。 So there's a bunch of other low level operations。

I mentioned the iocthal interface for changing， you know， resolutions and terminal。

and the bug rates and stuff。 There's ways of duplicating descriptors which you'll get to learn a lot about for your shell。

So for instance， I can take an old one and I can say take that file descriptor and take this new one and make that new one a duplicate of the old one。

et cetera。 There are pipes which are ways of communicating between processes you'll learn about those。

There's ways of locking files there's ways of memory mapping files so that you can share between processes。

Ways of doing asynchronous IO all sorts of stuff。 But the key thing here is that remember everything is a file。

Okay， in the way that I interact with it across the system call interface。

Everything has an open read right close pretty much everything。

And even if I'm talking to a network or I'm talking to a file or I'm talking to a pipe with another process all of those things I'm reading and writing is if it were a file and that's the。

that's the title of today's lecture is that everything is a file。 Yes， even a mouse is a file。 Okay。

you can open the open the stream on the slash dev part of the file system and there you can find a raw port to read from。

Okay， so what's the difference between high and low level API。

So I'm presenting him in the same lecture because I want you to always say does it have an F in front of it or not。

Is it an F open or is it an open am I mixing file stars with integers。

I want you to at least know that both of them are there so that you don't。

So you at least can hope to not mix them up。 Okay。

but if you look at the high level streaming interface。

What happens is the F read function is actually in lib C and has a whole bunch of stuff that runs at user level。

Like a normal function and it might occasionally make a system call where I have there's sort of assembly headers and stuff that you get to learn about in project one。

And then I call into the kernel which does some stuff for me and then I exit the kernel and I do more stuff at user level。

So I'm kind of like I'm wrapping this kernel interface with interesting user level stuff like buffering that I talked about in the streams。

And that's how I get a high level streaming reads rights opens F read F right F open。

The low level one is basically just kind of the raw interface to the kernel。

So this is the raw system call so I read as this raw system call。

an F read does a bunch of other stuff for buffering and so on。 Okay。

and so really the high level API is make it more useful for people by wrapping stuff around it。

Okay。 As Anthony basically is stating in the chat there。

you should take a look at slash damn slash it's got all sorts of weird files in there you can do all sorts of stuff with if you're the super user。

So I just want to do one last thing to keep in mind all the buffering so streams are buffered and user memory。

So for instance， print F being a streaming interface actually takes it basically takes a you know beginning of line that I sleep for a little while I say end of line。

And there's enough buffering in this that it waits at user level until the slash and which is the new line comes up and then it sends the。

the resulting line out to the screen。 And if I use the very low level things where for instance I write to the standard out beginning of line and sleep and then say end of line。

what will happen is you'll actually see beginning of line， and wait 10 seconds and say end of line。

And the reason for this is that there's buffering happening at user level in this case。 Okay。

So what I mentioned in the buffer cache inside the kernel fortunately is invisible to users。 Okay。

so it's there， and you occasionally have to flush to push something out to disk but it's mostly something you can can ignore the buffering at the user level with the streaming interfaces you have to be a little more careful about。

Okay， so I'm running administrative a little late today so my office hours。

I think for now we're going to do one to two on Tuesday and Wednesday。

So I picked a class day and a non class day in the hopes that people will be able to come to those。

And I may。 I don't know if I'll do zoom up I may post a zoom link。

We'll see how that goes my no next week in theory we're in the office so。

there's nobody left on the wait list so everybody that's going to be in the class is in the class。

And so tomorrow is drop deadline。 So you should make sure that if there are any friends of yours that might have been in the class and forgot they stayed in there but decided they were leaving。

Make sure they drop by tomorrow。 Okay， I every term， somebody at the end of the term comes and says。

Oh， Professor Kubie you know I forgot to drop it and now I'm got trouble trying to petition this and it doesn't always work。

So please make sure that they drop it and you drop if you're planning to drop。 Otherwise。

Otherwise you should start forming your groups。 Okay。

I would say you know who's going to be in the class。 You need groups of four。

We have the link for group formation up there and for saying kind of which sections you'd like to be in。

And so let's get that done。 Okay。 The other thing is if you notice carefully on the website and this is going to become more important as we go forward with more complicated stuff。

There are readings in your textbook that you should take a look at and I know that some people like to read as a way of really learning something in depth and so you know you should take a chance to try to read the chapters before the class to help you navigate some of the stuff we're going to be talking about。

Okay。 And you should be going to sections last week and this week， there are tomorrow on Friday。

because there is， there are some pieces of information that we're trying to give you in these early sections to get you ready to do ready to go on the projects and。

there are no assignments or restrictions on which section to go to so go for it next Friday。

we will have groups assigned and that point you'll have your section you should be going to。

and the group sign up is operational， all four of you in your group should be going to the same section or if you can't do that for some reason。

go to a section two sections that have the same TA。 Okay。 All right。 I think， and then midterm one。

two， three， we have all of the， the days are fixed。 The time is seven to nine。

And now it's just about dealing with conflicts and so on。

and we'll talk about other things as we go on in terms of conflicts。 I didn't put this on my slides。

but I do want to say as a matter of course we're opening up to reality right it's it's about time for us to be。

Doing things next week in person。 And so please if you're sick。

don't come to class okay don't go to your section and whatever we will have ways that will describe for you to make up for things that you might have missed。

etc。 Please don't come to class okay we want to make sure that everybody stays safe and healthy。

Okay。 And we will not have versions of， you know， checking for people attending and so on。

that would convince you that you need to come sick to class so if you feel like you're sick and you have to come to class。

Don't do it first you could ask your TA if there's something you're not sure about how to make it up。

Okay。 So I think that's all my administer V。 I know Anthony was going to give you some off of the office hours to。

I didn't manage to get them from him in advance but I will， we'll post them on the website and。

And he can post them in the chat if he likes。 All right。 So， so the going back。

There was a couple of questions on this last slide。 One is how are rights buffered。

So the answer about buffering rights is they're buffered both at the high level and the lower level。

The high level。 What happens is rights if you write a couple of characters they go into a buffer in in user memory。

And so they don't miss you know you write a couple of bytes to the file it may not even go to this until you flush and close。

Okay， so you got to be careful there because what the high level buffering is trying to do is eliminate too many cross。

Into the kernel and back because crossings into the kernel are expensive。

So that's one way things are written so you write little characters at a time with F right。

And then eventually you say a flush then it'll get pushed down into the the disk with a single F right。

Okay， this also happens inside the kernel and there it's more transparent to you。

and that you'll write a bunch of single characters with a system， call across each one of them。

They'll be buffered in inside the kernel and you'll need to make sure that they're flushed out to disk。

If you want to make sure they're on disk but other than that you won't know a lot about that lower level。

but you need to know the buffers there。 The other question is why does this delay so this is a buffered case here we print beginning of line goes into the buffer and user space and because print F is waiting for a slash and before it sends things out。

And nothing happens until you do an end of line and put the the backslash and which is a new line and then it'll actually go to the screen。

Okay。 So let's go a little lower。 Okay。 So we're going to look at the system call interface so I gave you some actual examples of system calls read right。

you know about for that's another system call。

So we were to look at the， for instance， the Linux system call reference there's a lot of them right so if you look here。

There's exit for read right open etc。 These are all numbered。 Okay。

why are the system calls numbered。 We have numbers for system calls rather than function pointers。

Yep。 So we do this because we're indexing into a vector。

because we cannot trust the user to say a generic function call。 Oh。

I know where forks address is turn it in。 So we need to get into the kernel switching into kernel mode but then starting at the beginning of a well defined vetted function like for a read。

And that's why everything's got a number。 Okay， and so the actual system call is a special type of synchronous trap that gives kind of this the call number。

And it looks gives looked up in a vector table。

Okay， so as far as the read and write interface that we've been talking about above the system call level we've been talking about descriptor numbers。

Okay。 Below， by the way， there's more than 255 system calls。

So we have a descriptor number which is a integer at the high level below。

There's actually a descriptor of the file that you've opened in a structure that's in kernel memory that the user can't look at。

Okay。 And so there's an integer at the top level and inside the descriptor is actually a structure that points at the file。

And this integer that the user knows about is indexed into a table to find this underlying structure。

Okay。 And we're not going to go in great detail。 But if you were to look in the Linux kernel or you look into Pintos kernel you'd see that there is a file structure。

And so， you know， the user can't look at it and for now there's at least two very important things so one is。

where is the file on disk if it's a file， file system disk file。

And that's an I know pointer and we're going to talk about that when we get into how file systems are made。

And that's the current position so every time you read a bite that that position inside the kernel advances automatically so that a subsequent read will be beyond which you've already gotten it so you can just do a bunch of reads to read from the beginning to the end of the file。

Okay。 Now， So， for system call to driver。 Let's look at what happens so when you execute the read system call。

It actually goes into the kernel。 It looks up the file structure for that read。

And then it calls some internal function like this VFS read will talk about the virtual file system interface later in the term。

So， we'll give you a flavor that here's this file star which we just showed you there that the kernel knows about。

And here the parameters the user asked about。 Okay。

read up to count bytes from the file put it into a buffer all of these things have been sanity checked and will be further sanity checked inside the kernel。

And then make sure you're allowed to read the file。

We make sure that the file has read methods okay so every file actually has a set of operations you're allowed to do and if you try to read to something that doesn't have a read option then you'll fail。

And you know what's an example of that well that might be a an output stream like a serial link you can't read from an output link and so it wouldn't have a read method and so if you try to read from it it would fail。

Also we check that the buffer has proper permissions because we don't trust the user to give us a good buffer。

We check whether we're reading from a valid range in the file。

And then we get to the actual reading。 Okay， and there's a table of functions and if the。

If that particular type of file if it's device driver allows you to do a synchronous read。

then it'll go ahead and use that function。 Otherwise it'll go and build a synchronous read out of the asynchronous read。

Okay， you don't have to worry about the all details I'm just giving you a flavor。 Okay。

and then there's a notification that says oh somebody read this and that's to the parent of that file。

And we'll update some bytes read by the current tasks that's a little bit of statistics and some other statistics and then we'll exit。

Okay， so when you call a system call read something like this is going to get executed in the kernel at kernel level。

Okay。

Now， let's look back here notice I mentioned this file。 This is a take the file itself pointer。

Look at its f op sub field and then look at the read inside of that。

This is actually the device driver for that type of file， whatever it is。

If it's on the file system， it might be a disk device driver， or it might be a file system。 Okay。

these are all lots of different options。 It could be referencing a mouse as people asked earlier。

Okay， so the device driver actually has a structure like this， called the file operations structure。

And every time a new device is put into the system。

A file operations is registered with the device driver that basically tells the kernel how to do all these things。

How do you read？ How do you write？ How do you do different things。 Okay。

So it's essential to our idea that everything's a file。

because the way we're able to make everything a file and have the same open read right。

close interface is every type of device we could care about every type of thing that we could try to。

read right close open from all register of file operation。

And so the kernel just can do the same thing。 You give it a read。

it'll go into that VFS read routine。 And it automatically knows which device driver to ask for data back from。

Is it the disk？ Is it your mouse？ Is it the network。

Okay。 So what is a device driver？ So device driver is this device specific code in the kernel that interacts directly with the device hardware。

It gives you this standard interface of open read right close。

And an I octal system call to do special things to the device。 Okay。

And device drivers typically are divided into two pieces which you'll also get to play with。

There's the top half， which comes in from the user in the system call level and gives you open close read right。

I octal strategy， etc。 And then there's the bottom half。

which is handling interrupts from the device itself。

And so all the interrupt handlers from that device will run in the bottom half。 Okay。 And I， again。

this is this lecture is about flavors of how this all kind of ties together。

If you think about read from a file here。 Okay。 What you see here is the user program is about to do a read。

And so it's going to request some IO by making a system call。 And that system call， if we do a read。

it's possible that the contents of that read are already cash in the kernels buffer cash。

in which case， if the answer is the data is already there。

then we'd have this quick path back out of the kernel。 We just copy stuff out of the cash。

adjust the pointers and move back up and all as well。 But if not。

we may need to ask the disk device to， to start a read， and we're going to put the process to sleep。

And what that really means is the kernel half of a user process gets put to sleep while the device is busy。

Okay， and we're going to talk a lot more about scheduling and putting things to sleep next time。

But imagine that if you have a user process is trying to read from disk that's a long latency operation。

And here's a number that hopefully by the end of the term you'll guys will have automatically but if you ask how many instructions does it take to do a disk read。

Okay。 Well， it depends a lot on the circumstances but a number to keep in mind is a million instructions worth of time。

Okay， to do a discrete that's a lot。 And so we don't want to waste all that time waiting for the disk and so the fact that we put the process to sleep。

mean somebody else can work。 Okay， and so at the top half of the device driver may put us to sleep。

And then we have set up the disk to read and so really the hardware is kind of implicitly monitoring and waiting for interrupts but somebody else is running。

And eventually， it interrupt comes and that interrupt handler will run and it will wake up the process that was sleeping。

Okay， and then it'll return from interrupt and the scheduler will take over and notice that that process should run again。

It'll copy the stuff out of the buffers from the device driver， et cetera。

It'll transfer to the user's buffers and complete。 Okay。 So files。

so the thing that this is glossing over file system for a moment。 Okay。

so think about this as reading from a device that doesn't have something complicated like a file system in it。

but rather， you know， like a raw disk or any other thing like that。

So this is this is handling read and remember everything looks like a file。

So it's read the read system call。 Once we get to the file system。

the file system is kind of what's in here with this， this blue kind of question mark here。 Okay。

so the idea of the file system and the file interface kind of gets handled up here。 Okay。

but we'll get to that later。 So， the last thing。 Okay， good。 I like that comment。

So there's comment in the chat。 So the OS isn't really pulling， pulling， I/O devices。

but kind of giving it an alert。 Yes。 What we do is we set up everything so that the process is。

is waiting on a weight cue for an event， which is an interrupt in this case from the device。

So it's sleeping on a weight cue associated with this device。

And the hardware itself will cause an interrupt that will wake that process up and take it off of the weight cue and put it back ready to run。

So yes， so the OS does not have to be polling to wait for devices in this way I've described it here。

Okay， so last but not least， let's push this idea that everything's a file even further。

So what about communication。 So， suppose that one process opens a file for writing and the other one opens it for reading。

Can we communicate between processes like that？ Certainly。 Right。 We can write to the file here。

And this guy can read from the file and data will go through the file system and get transferred to the other guy。

And we've just communicated between two processes。 Now。

what if we only write data once and read it only once。 So this is really like a cue。

And this is a little wasteful because we're filling up the disk with something that isn't going to be read again。

Right。 Because this is a cue。 And so then we can start thinking about a cue。 And in fact。

the typical example you like to think about is the network。

So a good example of that is we have cues on either side of the network。

And we set them up and the client writes to that cue。

And it goes across the network into another cue on the server side and the server reads from it。

Okay， so these are connected cues over the Internet。

But notice the interface is still looks like a file。 Okay。

we're reading and writing file descriptors。 So the question might be。

what's the analog of open here。 All right。 What's the name space。 How are they connected in time。

All of this stuff。 We're going to have to figure out。 But the simple idea is everything's a file。

And in fact， we can go more with this。 Right。 We can think of a client server on a single machine where the client issues a right of a request。

Okay， and then it waits for the response。 The server running on the same machine reads that request services it sends it back。

And we through a read， we wake up and go forward。 Okay。

so this would be fine on a single machine but voila， let's put the network in there。 Okay。 Now。

that sounds good。 All right， this is starting to look like like a web server so far。 Right。

So the socket abstraction is really this idea of a cue that goes across from goes across from one endpoint to another。

And those endpoints can actually span the network。 Okay。 So for instance。

when we have sockets on either side， there can actually be the network in here。

And one process opens a socket on one side。 The other process opens on the other side。

And we have some way of connecting the two。 And if we do that。

then we'll be able to write on one side and read on the other and it will just work。 Okay。

Now sockets are this idea of an endpoint for communication。

which are cues for temporarily holding results。 These cues don't have to be across the network。

They could be on the same machine。 They could be across the network。 And in fact， we could。

we could have the same two client server kind of sets of code。

one of which is running at one point on the same machine and another point on different machines。

And we're going to talk a lot about that level of flexibility later in the term as well。

So two sockets connected over the internet or over a network gives us inter procedural or inter process communication over the network。

Okay， and we still don't know how to open or what the name space is。 Okay。

And a good question in the chat is， well， if I go to do a read and there isn't anything there。

what happens。 The answer is yes， it'll get put to sleep until the data comes。 Okay。 Exactly。

All right。 Now， right now， this is actually。 So the question in the chat is how does it contrast if we were trying to do this without everything being a file。

So， in Unix， you don't do this without everything being a file。 Okay， here in Unix。

this is exactly the interface。 And this， in this case， the sockets are not storing stuff on disk。

they're just cues that are passing from one across the network to the other。

But the interface that's right and read look exactly like the file interface and that's the idea that everything's a file is the interface is the same。

Okay。 Can two processes share a socket to processes can have two ends of a socket。 Okay。

we're going to look at different ways of communicating later。 Okay。

so more details so a socket's an abstraction for one endpoint of a network connection， for instance。

most operating systems provided， even if they don't have the rest of the Unix I， I， O， API。 Okay。

they were first introduced in BSD Unix。 Okay。 And 4。2 and what was interesting was 4。

2 BSD was such a big deal。 This was in the days before everybody had the internet in the way that we're talking about it now that what happened was there were at Berkeley。

they were actually producing the 4。2 release the final version of it copying it on tapes。

And there were runners sitting at Berkeley waiting to get the tapes to run them back to their companies。

And， you know， and load them onto their machines。 So this was a really lots of excitement with 4。

2 BSD sockets were just one of the many things there。 Okay。

And it's the same abstraction for lots of types of network， including no network。

So it's local within the machine， the internet TCP IP UDP IP work with sockets。

and even things that nobody uses anymore。 There's types of networking like OSI and Apple Talk and IPX and all of these things that also use sockets。

Okay。 So it looks just like a file with a file descriptor with read and write。 Okay。 Now。

not all the， not all the file type calls make sense like you can't else seek on a socket。 Okay。

but you can read and write to it。 And how can you use sockets to give you real applications？ Well。

it's a byte stream。 And the real applications typically have to wrap something else on top of it。

like a messaging facility or a remote procedure call facility that can basically serialize and encode things。

et cetera。 And we will， we'll spend time talking about that later in the term as well。 But for now。

we're just communicating bytes。 So here's a simple example of an echo server。

Clients is hello world web server goes hello world。 Okay， kind of a silly one。 Here's how it looks。

Here's the two sockets。 Let's assume they've been connected。 So at the client， we're going to echo。

But before that happens， the server starts up and goes to sleep on a week on a read。

That's a blocking read。 The client potentially this F get S is actually getting a string from the user that the user is going to type hello world。

And then it will use that resulting buffer， write it to the socket。

It'll then go and execute a read， which is going to cause it to wait。 So it's waiting for the echo。

So it wrote the value out to the network and it's going to wait。 And meanwhile。

the data goes out from the sockets to the server side。 The server is going to take it。

It's going to print it on its own screen。 It's going to send it back。 So go across the network。

it'll wake up the client and voila， we've just echoed and then we will repeat on both sides。 Okay。

Now， good question。 It's a string guaranteed to come back in chunks， all at once or in chunks。

In fact， it's not guaranteed to come back all at once。

And so you actually have to execute read multiple times potentially to look for that null at the end。

Good question。 So here's a fairly simple， really quick set of code。

I'm assuming the sockets are already open because I haven't shown you how to do that yet。

But here we have the right on one side and the read on the other and notice I'm not checking in a loop like I should with read。

That's a good question。 But then I take the result and assuming there are no errors。

I write it back and I get my echo。 Okay， echo echo echo。

So what assumptions are we making here we're assuming it's reliable。 So when you write to it。

you get it back。 And so there's nothing lost。 And so that's a special type of socket。

We need to set up a TCP socket to the other side。 We're assuming everything's in order so that gets helped by a TCP socket as well。

And when are things ready？ Well， we're going to rely on read going to sleep。

If there's no bytes ready。 So how do we create sockets。 Well。

file systems basically provide a collection of permanent objects。 And， you know。

so it's easy to open a file。 But sockets are a little funny， right？

So it's a two way communication between processes on completely different machines。

So we have to have some way to rendezvous between our desire to make a connection and somebody's remote desire to provide service。

And so that's going to be a question of how do we name the objects were opening。

And how do these completely independent programs know that the other one wants to talk to them。

Okay， and I'm going to finish this up but let's just keep in mind。 What is the namespace here。

And we have file systems。 The namespace is just the set of all possible path names right。

In the case of communicating over the network the day space is something like host names or IP addresses。

And also port numbers so given IP address represents a machine。 So port number represents a service。

Okay， and well， so typically if you're connecting to somebody you're actually connecting to a combination of the IP address and port。

All right， and so this is the last thing we're going to talk about today。

Bear with me for two slides。 So here's how we actually set up a socket。

And the server side starts by creating what's called a server socket。

And that server socket executes a listen。 And that listens as I'm listening for somebody who wants to connect it my IP address with this port that everybody knows about。

Could be the echo port。 It could be the web server port。 Okay。

later the client does a connection request saying hi here's my address here's a port I've got。

I'd like to connect。 If the circuit the server socket accepts。

then it makes a brand new socket with a connection。 Okay。

so basically that connection is now a set of connected sockets。 And that's the green ones。

And that yellow connection is actually defined uniquely by a five tuple。

What is that five tuple the source IP address， the destination IP address。

the source port number the destination port number， and the， protocol which in this case is TCP。

Okay， the client port which is needed for uniqueness is often randomly chosen。

And that port which is well known is typically something like 80 for the web or 443， etc。 Okay。

All right， so we need to finish。 So in conclusion the system call interface is a narrow waste between user programs in the kernel。

Streaming IO is a stream of bytes。 Most of the streaming functions start with F like F3。

And we have a little bit of a level IO which is the actual kernel interface。 Okay， system calls。

We talked about composition。 We talked about what the device drivers are and how they work。

And the file abstraction is basically used for everything。 Okay。

and then we finally finished up with sockets which will pick up next time。 Sorry， I went over。

I hope you all have a good evening。 And we will see you later。 Ciao。 [ Silence ]。