位置: IT常识 - 正文

6.824 Lab 1: A simple web proxy

编辑:rootadmin
6.824 Lab 1: A simple web proxy6.824 - Spring 20046.824 Lab 1: A simple web proxyDue: Tuesday, Febru

推荐整理分享6.824 Lab 1: A simple web proxy,希望有所帮助,仅作参考,欢迎阅读内容。

文章相关热门搜索词:,内容如对您有帮助,希望把文章链接给更多的朋友!

6.824 - Spring 20046.824 Lab 1: A simple web proxyDue: Tuesday, February 10th, 1:00pm.Introduction

Please read Getting started with 6.824 labs before starting this assignment. You will also need Using TCP through sockets at a later stage.

If you have questions, please first read Office hours and asking questions. After you have done that, you can send e-mail to 6.824-staff@pdos.lcs.mit.edu.

In this lab assignment you will write a simple web proxy. A web proxy is a program that reads a request from a browser, forwards that request to a web server, reads the reply from the web server, and forwards the reply back to the browser. People typically use web proxies to cache pages for better performance, to modify web pages in transit (e.g. to remove annoying advertisements), or for weak anonymity.

You'll be writing a web proxy to learn about how to structure servers. For this assignment you'll start simple; in particular your proxy need only handle a single connection at a time. It should accept a new connection from a browser, completely handle the request and response for that browser, and then start work on the next connection. (A real web proxy would be able to handle many connections concurrently.)

In this handout, we use client to mean an application program that establishes connections for the purpose of sending requests[3], typically a web browser (e.g., lynx or Netscape). We use server to mean an application program that accepts connections in order to service requests by sending back responses (e.g., the Apache web server)[1]. Note that a proxy acts as both a client and server. Moreover, a proxy could communicate with other proxies (e.g., a cache hierarchy).

Design Requirements

Your proxy will speak a subset of the HTTP/1.0 protocol, which is defined in RFC 1945. You're only responsible for a small subset of HTTP/1.0, so you can ignore most of the spec. You should make sure your proxy satisfies these requirements:

GET requests work.Images/Binary files are transferred correctly.Your webproxy should properly handle Full-Requests (RFC 1945, Section 4.1) up to, and including, 65535 bytes. You should close the connection if a Full-Request is larger than that.You must support URLs with a numerical IP address instead of the server name (e.g. http://18.181.0.31/).You are not allowed to use fork().You may not allocate more than 100MB of memory.You can not have more than 32 open file descriptors.Your proxy should correctly service each request if possible. If an error occurs, and it is possible for the proxy to continue with subsequent requests, it should close the connection and then proceed to the next request. If an error occurs from which the proxy cannot reasonably recover, the proxy should print an error message on the standard error and call exit(1). There are not many non-recoverable errors; perhaps the only ones are failure of the initial socket(), bind(), listen() calls, or a call to accept(). The proxy should never dump core except in situations beyond your control (e.g. a hardware or operating system failure).

You do not have to worry about correct implementation of any of the following features; just ignore them as best you can:

POST or HEAD requests. URLs of any type other than http. HTTP-headers (RFC 1945, Section 4.2).

If your browser can fetch pages and images through your proxy, and your proxy passes our tester (see below), you're done.

HTTP example without a web proxy

HTTP is a request/response protocol that runs over TCP. A client opens a connection to a web server and sends a request for a file; the server responds with some status information and the file contents, and then closes the connection.

You can try out HTTP yourself:

% telnet web.mit.edu 80

This connects to web.mit.edu on port 80, the default port for HTTP (web) servers.

Then type

GET / HTTP/1.0

followed by two carriage returns. This ends the header section of the request. The server locates the web page and sends it back. You should see it on your screen.

To form the path to the file to be retrieved on a server, the client takes everything after the machine name. For example, http://web.mit.edu/resources.html means we should ask for the file /resources.html. If you see a URL with nothing after the machine name and port, then / is assumed---the server figures out what page to return when just given /. Typically this default page is index.html or home.html.

On most servers, the HTTP server lives on port 80. However, one can specify a different port number in the URL. For example, typing http://web.mit.edu:2206 in your browser will tell it to find a web server on port 2206 on web.mit.edu. (No, this doesn't work for this address.)

HTTP (request) example with a web proxy

Before you can do this example, you need to tell your web browser to use a web proxy. This explanation assumes you are running Mozilla, but things should be remarkably similar for Netscape. Choose ``Edit'' ---> ``Preferences''. Then choose ``Advanced'' ---> ``Proxies''. Click on ``Manual proxy configuration''. Now set the ``HTTP proxy'' to speakeasy-mit-ron.lcs.mit.edu and port 3128. Mozilla will now send all HTTP request to this web proxy rather than directly to web servers.

Lynx---a poor man's browser---can be told to use this web proxy by setting the environment variable http_proxy to speakeasy-mit-ron.lcs.mit.edu:3128.

Now to the real stuff.

You can use nc to peek at HTTP requests that a browser sends to a web proxy. nc lets you read and write data across network connections using UDP or TCP[10]. The class machines have nc installed.

First we'll examine the requests that a browser sends to the proxy. We'll use nc to listen on a port and direct our web browser (Lynx) to use that host and port as a proxy. We're going to let nc listen on port 8888 and tell Lynx to use a web proxy on port 8888.

% nc -lp 8888

This tells nc to listen on port 8888. Chances are that you will have to choose a different port number than 8888 because someone else may be using that port. Choose a number greater than 1024, less than 65536. Now try, on the same machine, to retrieve a web page port 8888 as a proxy:

% env http_proxy=http://localhost:8888/ lynx -source http://www.yahoo.com

This tells Lynx to fetch http://www.yahoo.com using a web proxy on port 8888, which happens to be our spy friend nc.

Netcat neatly prints out the request headers that Lynx sent:

% nc -lp 8888GET http://www.yahoo.com/ HTTP/1.0Host: www.yahoo.comAccept: text/html, text/plain, application/vnd.rn-rn_music_package, application/x-freeamp-theme, audio/mp3, audio/mpeg, audio/mpegurl, audio/scpls, audio/x-mp3, audio/x-mpeg, audio/x-mpegurl, audio/x-scpls, audio/mod, image/*, video/mpeg, video/*Accept: application/pgp, application/pdf, application/postscript, message/partial, message/external-body, x-be2, application/andrew-inset, text/richtext, text/enriched, x-sun-attachment, audio-file, postscript-file, default, mail-fileAccept: sun-deskset-message, application/x-metamail-patch, application/msword, text/sgml, */*;q=0.01Accept-Encoding: gzip, compressAccept-Language: enUser-Agent: Lynx/2.8.4rel.1 libwww-FM/2.14 SSL-MM/1.4.1 OpenSSL/0.9.6b

The GET request on the first tells the proxy to get file http://www.yahoo.com using HTTP version 1.0. Notice how this request is quite different from the example without a web proxy! The protocol and machine name (http://www.yahoo.com) are now part of the request. In the previous example this part was omitted. Look in RFC 1945 for details on the remaining lines. (It's effective reading material if you really can't sleep and Dostoevsky didn't do the trick.)

HTTP (reply) example with a web proxy

The previous example shows the HTTP request. Now we'll try to see what a real web proxy (speakeasy-mit-ron.lcs.mit.edu port 3128) sends to a web server. To achieve this we use nc to be a fake web server. Start the ``fake server'' on anguish.lcs.mit.edu with the following command:

% nc -lp 88886.824 Lab 1: A simple web proxy

Again, you may have to choose a different number if 8888 turns out to be taken by someone else.

% env http_proxy=http://speakeasy-mit-ron.lcs.mit.edu:3128/ lynx -source http://anguish.lcs.mit.edu:8888

Needless to say, you should replace 8888 by whatever port you chose to run nc on. nc will show the following request:

% nc -lp 8888GET / HTTP/1.0Accept: text/html, text/plain, audio/x-pn-realaudio, audio/vnd.rn-realaudio, application/smil, text/vnd.rn-realtext, video/vnd.rn-realvideo, image/vnd.rn-realflash, application/x-shockwave-flash2-preview, application/sdp, application/x-sdpAccept: application/vnd.rn-realmedia, image/vnd.rn-realpix, audio/wav, audio/x-wav, audio/x-pn-wav, audio/x-pn-windows-acm, audio/basic, audio/x-pn-au, audio/aiff, audio/x-aiff, audio/x-pn-aiff, text/sgml, video/mpeg, image/jpeg, image/tiffAccept: image/x-rgb, image/png, image/x-xbitmap, image/x-xbm, image/gif, application/postscript, */*;q=0.01Accept-Encoding: gzip, compressAccept-Language: enUser-Agent: Lynx/2.8.4rel.1 libwww-FM/2.14Host: anguish.lcs.mit.edu:8888X-RAN-Loopstop: trueX-RAN-Loopstop: trueVia: 1.0 speakeasy.ron.lcs.mit.edu:3128 (squid/2.5.STABLE2), 1.0 speakeasy.ron.lcs.mit.edu:3148 (squid/2.5.STABLE4), 1.0 nyu.ron.lcs.mit.edu:3128 (squid/2.5.STABLE4)X-Forwarded-For: 18.26.4.9, 127.0.0.1, unknownCache-Control: max-age=259200Connection: keep-alive

Notice how the web proxy stripped away the http://anguish.lcs.mit.edu:8888 part from the request!

Your web proxy

Your web proxy will have to translate between requests that the client makes (the one that starts with ``GET http://machinename'') into requests that the server understands. So far for the bad news. The good news is that we provide you with some helpful code that will make this very easy to do.

Your web proxy will listen on a port other than port 80, so as to avoid conflicts with regular web servers.

Once the request line has been received, the web proxy should continue reading the input from the client until it encounters a blank line. The proxy should then fetch the URL from the appropriate server, forward the response back to the client, and close the connection. The proxy should forward response data as it arrives, rather than buffering the entire response; this allows the proxy to handle huge responses without running out of memory.

Your web proxy has to support the GET method only [3]. A GET method takes two arguments: the file to be retrieved and the HTTP version. Additional headers may follow the request.

Getting Started

We have provided a skeleton webproxy directory. It is available at http://pdos.lcs.mit.edu/6.824/labs/webproxy1.tar.gz. The following sequence of commands should yield a compiled version of the server you should extend to pass the tests.

% wget http://pdos.lcs.mit.edu/6.824/labs/webproxy1.tar.gz% tar xzvf webproxy1.tar.gz% cd webproxy1% gmake

The tarball contains http.C, http.h, Makefile, webproxy1.C and webproxy1-test.C. The first two files will help you parse HTTP requests. The Makefile is, as its meaningful name implies, a Makefile. Webproxy1.C is a pretty useless web server that, nonetheless, should help you on your way. webproxy1-test.C is our testing program which checks your program for correctness.

http.C and http.h : a HTTP parser

We have provided a parser for proxy-style HTTP requests. It is implemented in the files http.C and http.h that are included in the tarball.

http.h defines the class httpreq that inherits from the class httpparse (if you are unfamiliar with C++ inheritance, consult the Stroustrup C++ language guide referenced in the course information page. Don't drop this book on someone's face. It's a pretty hefty book.)

To parse a request, first create a httpreq object. Then, parse the (potentially incomplete) HTTP request by feeding it to int parse (char *buf, ssize_t len) until it returns 1, indicating that the headers are complete. buf should be the buffer that contains the (potentially incomplete) HTTP request. len is the length of the HTTP request fragment in buf. Notice that parse needs to see the whole request you have read so far.

parse returns 1 if the HTTP request is complete, 0 if it needs more data to complete, or -1 on a parse error. parse does not modify the contents of buf. Once parse returns 1, you can call---amongst others---the following methods on the calling httpreq.

char* method() The 'type' of request (POST, GET, HEAD)char* host() The destination hostshort port() The destination portchar* path() The filename part of the requested URLchar* url() The requested URL

Here's a simple program that illustrates the use of httpreq.

#include <stdio.h>#include "http.h"intmain(){ httpreq *r = new httpreq(); char buf[512]; int ret; // incomplete header strcpy(buf, "GET http://web.mit.edu/index.html"); ret = r->parse(buf, strlen(buf)); printf("ret %d file %s\n", ret, ret == 1 ? r->path() : "(none)"); // complete header strcat(buf, " HTTP/1.0\r\n\r\n"); ret = r->parse(buf, strlen(buf)); printf("ret %d file %s\n", ret, ret == 1 ? r->path() : "(none)"); delete r; exit(0);}Documentation

You may want to read Using TCP through sockets to learn about socket programming in C/C++. Also, take a look at the references at the bottom of this page.

Running and testing the proxy

Your proxy program should take exactly one argument, a port number on which to listen. For example, to run the proxy on port 2000:

% ./webproxy1 2000

As a first test of the proxy you should attempt to use it to browse the web. Set up your web browswer to use one of the class machines running your proxy as a proxy and experiment with a variety of different pages.

When you think your proxy is ready, you can run it against the test program webproxy1-test, our tester. Run the tester with your proxy as an argument:

% ./webproxy1-test ./webproxy1

Note that this may take several minutes to complete. The test program runs the following tests:

Ordinary fetch

This test is the "normal case". We send a normal HTTP 1.0 GET request and expect the correct web page.

Split request

This tests splits the HTTP request in two chunks. The first chunk contains a partial HTTP request. The second chunk completes the first after which the tester expects the correct web page contents to come back.

Large request

The tester does a request of exactly 65535 bytes.

Large response

The tester fetches a web page larger than the maximum amount of memory available to your web proxy.

Zero-size response

The tester fetches a web page without a body.

Recover after bad connect

The tester sends a request with a URL that specifies a false port. Your proxy will attempt to make a connection to a bogus port. Soon thereafter, the tester tries to fetch a valid page to see if your proxy is still doing ok.

Malformed request

The tester sends an HTTP request that is not syntactically correct. After that, it tries to fetch a valid page to see if it your proxy is still doing ok.

Premature client close()

The tester sends a partial HTTP request and then closes the connection. After that, it tries to fetch a valid page to see if it your proxy is still doing ok.

Infinitely long request

The tester swamps your proxy with a request larger than 65535 bytes. The tester expects your proxy to close the connection. After that, it tries to fetch a valid page to see if it your proxy is still doing ok.

Stress test

The tester stress tests your web proxy with a ruthless combination of ordinary fetches, split requests, malformed requests, and large responses. This may expose memory leaks, unclosed connections, and random other bugs.

Collaboration policy

You must write all the code you hand in for the programming assignments, except for code that we give you as part of the assigment. You are not allowed to look at anyone else's solution (and you're not allowed to look at solutions from previous years). You may discuss the assignments with other students, but you may not look at or copy each others' code.

Handin procedure

You should hand in a gzipped tarball webproxy1-handin.tgz produced by gmake dist. Copy this file to ~/handin/webproxy1-handin.tgz. Do not make this file world readable! We will use the first copy of the file that we can find after the deadline---we try every few minutes. Don't bother to copy a new version over the old one hoping that we will use it instead. We won't.

References1Apache Web Proxy, http://www.apache.org/docs/mod/mod_proxy.html.2T. Berners-Lee, et al. RFC 1945: Hypertext Transfer Protocol - HTTP/1.0, May 1996.3CERN Web Proxy, http://www.w3.org/Daemon/User/Proxies/Proxies.html.4Netcat. http://www.atstake.com/research/tools/nc110.txt.
本文链接地址:https://www.jiuchutong.com/zhishi/303719.html 转载请保留说明!

上一篇:CSS Houdini:用浏览器引擎实现高级CSS效果(css代码怎么在浏览器运行)

下一篇:帝国cms如何投稿(帝国cms怎么上传图片)

  • 安卓怎么清除内存(安卓怎么清理内存)(安卓怎么清除手机内存存储空间)

    安卓怎么清除内存(安卓怎么清理内存)(安卓怎么清除手机内存存储空间)

  • 魅族mx3配置(魅族mx3怎么样)(魅族mx3hifi)

    魅族mx3配置(魅族mx3怎么样)(魅族mx3hifi)

  • iqoo8pro怎么设置通知栏样式(iQOO8pro怎么设置双系统)

    iqoo8pro怎么设置通知栏样式(iQOO8pro怎么设置双系统)

  • qq空间怎么注销(qq空间怎么注销掉手机版)

    qq空间怎么注销(qq空间怎么注销掉手机版)

  • 钉钉无法打卡怎么处理(钉钉无法打卡怎么办啊)

    钉钉无法打卡怎么处理(钉钉无法打卡怎么办啊)

  • 华为防频闪有必要开吗(华为防频闪开吗)

    华为防频闪有必要开吗(华为防频闪开吗)

  • 表格里的文字怎么全部显示出来(表格里的文字怎么提取出来)

    表格里的文字怎么全部显示出来(表格里的文字怎么提取出来)

  • 漫步者tws1蓝牙耳机怎么配对(漫步者TWS1蓝牙耳机)

    漫步者tws1蓝牙耳机怎么配对(漫步者TWS1蓝牙耳机)

  • 微信怎么能更改微信号(微信怎么能更改好友头像)

    微信怎么能更改微信号(微信怎么能更改好友头像)

  • 抖音背包礼物能提现吗(抖音背包礼物能送给别人背包么)

    抖音背包礼物能提现吗(抖音背包礼物能送给别人背包么)

  • 表格有虚线怎么变实线(表格有虚线怎么弄掉)

    表格有虚线怎么变实线(表格有虚线怎么弄掉)

  • 华为手机无限重启震动(华为手机无限重启开不了机怎么办)

    华为手机无限重启震动(华为手机无限重启开不了机怎么办)

  • 12306封ip多久会恢复(12306封ip解除)

    12306封ip多久会恢复(12306封ip解除)

  • facetime和微信哪个费流量(facetime和微信哪个好用)

    facetime和微信哪个费流量(facetime和微信哪个好用)

  • 华为手机插件设置方法(华为手机如何调出插件)

    华为手机插件设置方法(华为手机如何调出插件)

  • 安卓机怎么把数据移到另一只手机(安卓机怎么把数据移到安卓手机)

    安卓机怎么把数据移到另一只手机(安卓机怎么把数据移到安卓手机)

  • 滴滴出行如何修改目的地(滴滴出行如何修改车辆信息)

    滴滴出行如何修改目的地(滴滴出行如何修改车辆信息)

  • 通常所有的主机是指(通常所说的主机主要指?)

    通常所有的主机是指(通常所说的主机主要指?)

  • 荣耀手环3接电话按哪里(荣耀手环接电话怎么接)

    荣耀手环3接电话按哪里(荣耀手环接电话怎么接)

  • 拼多多猫好友怎么删除(拼多多怎么互相加为好友)

    拼多多猫好友怎么删除(拼多多怎么互相加为好友)

  • 三星s10和s9+对比(三星s10和三星s9+哪个好)

    三星s10和s9+对比(三星s10和三星s9+哪个好)

  • 电视信号源怎么切换(电视信号源怎么选)

    电视信号源怎么切换(电视信号源怎么选)

  • Win10如何建立多个桌面 Win10建立多个桌面的方法(win10怎么建立多个用户)

    Win10如何建立多个桌面 Win10建立多个桌面的方法(win10怎么建立多个用户)

  • win11电脑怎么设置鼠标滑轮一次滚动一个屏幕?(win11电脑怎么设置开机密码)

    win11电脑怎么设置鼠标滑轮一次滚动一个屏幕?(win11电脑怎么设置开机密码)

  • python如何转移数据库里的数据(python转换数字)

    python如何转移数据库里的数据(python转换数字)

  • 个人所得税申报操作流程2023
  • 免税外汇商品
  • 背书承兑分录
  • 其他应收款怎样做平
  • 企业所得税1季度申报季初从业人数个税所属期12月
  • 固定资产转让开票金额是净值
  • 零申报小微企业注销有影响吗
  • 捐赠的固定资产账务处理
  • 关于年底双薪和分红问题的处理
  • 政府扶持资金科目怎么做?
  • 800多项进口关税减免
  • 现金账冲账怎么冲
  • 普通发票发票联丢了
  • 个人所得税申报是什么意思?
  • 制作费计入什么会计科目
  • 买二手车年限久的会怎样
  • 小规模纳税人金额
  • 土地收储的含义
  • 长期股权投资是什么科目
  • 事业单位财政拨款是什么意思
  • 开票金额比收入高怎么办
  • 以前年度损益调整
  • 现金分红说明什么
  • 外包公司代缴的税怎么算
  • 电脑每次开机都要选择系统怎么办
  • window10没有激活码
  • linux 数据类型
  • 注销公司账上现金多怎么调
  • 罚款收入营业外收入还是其他应收款
  • 手把手怎么样
  • 最贵的节能灯泡品牌
  • 关于交易性金融资产的问题
  • php 时间格式转换
  • 限售股转让个人所得税退税
  • 前端实现微信联系人二维码
  • 对公账户转入对私账户
  • 失控增值税专用发票
  • 低代码开发开源
  • Access-Control-Allow-Origin 翻译
  • php连接不上数据库sql
  • zend framework手册
  • vue的路由守卫有哪些钩子函数
  • 资产负债表日后调整事项
  • 股权划转有税费吗
  • Vite + Vue2 + Vuetify2 + <script setup> + TypeScript 搭配开发项目
  • SQLite教程(四):内置函数
  • 以房抵债的税会退税吗
  • 不动产销售额是否为免税
  • 支付所得税属于经营活动产生的现金流量吗
  • sql server 2008 r2教程
  • 财务报表与分析outcome2
  • 合作社收到政府补贴会计分录
  • 网络服务费计入什么费用
  • 长期挂账其他应付款处理
  • 建安企业增值税预缴
  • 工厂没给工人买社保我们应该找哪个部门
  • ca证书费用计入什么科目
  • 处置长投其他权益变动要转损益吗
  • 文化事业费的费率是多少?
  • 固定资产折旧方法的选择
  • 固定资产确认条件最新
  • 各种会计凭证的填制
  • win10下mysql 5.7.17 zip压缩包版安装教程
  • Win2003下APACHE+PHP5+MYSQL4+PHPMYADMIN 的简易配置方法
  • mac os图片
  • 七喜r11c笔记本
  • linux忘记root密码怎么办
  • windows7桌面右击
  • opera installer
  • 深入分析的成语
  • nodejs怎么开发后端
  • ubuntu nfs配置
  • mare的用法
  • jQuery使用Selectator插件实现多选下拉列表过滤框(附源码下载)
  • 广东省电子税务局app下载手机版
  • 企业个税网上申报时间
  • 电子税务平台怎么红冲纸质发票
  • 广州地税微信公众号
  • 预付建造固定资产的工程价款
  • 注册会计师人员名单
  • 免责声明:网站部分图片文字素材来源于网络,如有侵权,请及时告知,我们会第一时间删除,谢谢! 邮箱:opceo@qq.com

    鄂ICP备2023003026号

    网站地图: 企业信息 工商信息 财税知识 网络常识 编程技术

    友情链接: 武汉网站建设