Tuesday, March 1, 2011

http request methods


Hello every reader,
as I stated before that I will explain each word in the Http request or response,starting from the first word of the Http request which is the request method.
There are eight different methods of Http requests and thy are:
OPTIONS
GET
HEAD
POST
PUT
DELETE
TRACE
CONNECT

NOTE:first take my advice and install apache (or any Http server)to your system,then come back to lesson.

OPTIONS:this method is used to request information about the allowed request methods associated with a the requested resource and supported by the server,the client recognizes those allowed methods,because they will be preceded by “Allow” header,for example let's send an OPTIONS request to localhost or any host you want,to send an Http request there many ways,let's follow this:
first open your command line and type:
telnet localhost 80
this means open a connection to the host “localhost” on the port 80(which is the Http port, here is list to common ports).
After typing this you see a message telling you that you are connected to localhost,just like this:
Trying ::1...
Connected to localhost.
Escape character is '^]'.

after this message you must type the Http request,you know that the first line is a must but the others are optional,let's write the OPTIONS request:
OPTIONS / HTTP/1.0
after typing the previous line leave a blank line by pressing enter then re-press enter again(the first enter is for leaving a blank line and the second enter is for sending the request to localhost).
If your request succeeds you will see the following response:
HTTP/1.1 200 OK
Date: Mon, 21 Feb 2011 18:22:57 GMT
Server: Apache/2.2.14 (Ubuntu)
Allow: GET,HEAD,POST,OPTIONS
Vary: Accept-Encoding
Content-Length: 0
Connection: close
Content-Type: text/html

I think you see that the Allowed methods are:
GET,HEAD,POST,OPTIONS

GET:this method is used to get the content of the requested resource such as an image,html,and many other common http mime types described by CONTENT-TYPE header,the GET request can carry more information as parameters,for example let's have the following page new.php,we will send this page a GET request with two parameters,word1 and word2:

<?php
$word1=$_GET['word1'];
$word2=$_GET['word2'];
?>
<html>
<body>
<?php
echo $word1.” “.$word2;
?>
</form>
</body>
</html>

this request looks like this:
GET /new.php?word1=hello&word2=world HTTP/1.0

the response will be:
HTTP/1.1 200 OK
Date: Mon, 21 Feb 2011 18:58:09 GMT
Server: Apache/2.2.17 (Win32) PHP/5.3.5
X-Powered-By: PHP/5.3.5
Content-Length: 43
Connection: close
Content-Type: text/html

<html>
<body>
hello world</body>
</html>

HEAD:this method is similar to GET but the response will be without the message body,(we haven't talked about the Http response yet,but each response contains a Message Body as we will see later).
This method is usually used to test hypertext links for validity(the link works or not),accessibility(you have the permission to access the requested resource or not) and recent modifications.
Let's now send a HEAD request to localhost and compare it with GET request:
HEAD / HTTP/1.0

the response will be:
HTTP/1.1 200 OK
Date: Fri, 25 Feb 2011 08:39:56 GMT
Server: Apache/2.2.14 (Ubuntu)
Last-Modified: Tue, 01 Feb 2011 08:31:24 GMT
ETag: "e041c-b1-49b345b95ef00"
Accept-Ranges: bytes
Content-Length: 177
Vary: Accept-Encoding
Connection: close
Content-Type: text/html

now send a GET request to local host requesting / and see the response.

POST:this method is used to send data to the server with capability to carry more data than GET
with more secrecy (the sent information is enclosed inside the POST_DATA portion of the Http request whereas in GET the sent information is always sent within the Http_url).
POST can be used also to upload files to web servers by sending the file content in the POST_DATA.
Example:
you can send POST request as in the previous examples.
Examples:
this is an example of a POST request issued by a form with two text fields as the above GET request:
POST /new.php HTTP/1.0
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/x-shockwave-flash, application/xaml+xml, application/vnd.ms-xpsdocument, application/x-ms-xbap, application/x-ms-application, application/msword, application/vnd.ms-powerpoint, */*
Accept-Language: ar-sa
Content-Type: application/x-www-form-urlencoded
Pragma: no-cache
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; GTB6.6; Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) ; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30; .NET CLR 1.1.4322; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022)
Host: localhost
Proxy-Connection: Keep-Alive
Content-Length: 25

word1=hello&word2=world

the response is:
HTTP/1.1 200 OK
Date: Fri, 25 Feb 2011 14:28:14 GMT
Server: Apache/2.2.17 (Win32) PHP/5.3.5
X-Powered-By: PHP/5.3.5
Content-Length: 45
Connection: close
Content-Type: text/html

<html>
<body>
hello world
</body>
</html>

PUT:this method is used to upload files to web servers in simpler way than POST requests,example of uploading a file using PUT request:
PUT /script.php HTTP/1.1
the previous example will upload the file script.php to the web server,but this method is disabled in most web servers because the attackers can exploit this method to overwrite the original files located at that server.

DELETE: we use DELETE request to delete the resource specified by the request ,the DELETE request looks like:
DELETE /file.ext HTTP/1.0
this request deletes the file file.ext from the web server,this method is disabled in most web servers
because it can be exploited by bad people to delete what they want to delete.

NOTE:you can configure your web server to redirect those requests to a specified script.

TRACE:it allows the client to see what is being received by the other end of the request chain and
use that data for testing and diagnostic information,the request chain may be:
client->proxy->server or client->server.
I mean that the other end responds to the client with the last request it receives,this enables the client to know if there is a proxy in the middle of the way between the client and server,I'm sorry but I have no real world examples.

CONNECT:This specification reserves the method name CONNECT for use with a
proxy that can dynamically switch to being a tunnel (e.g. SSL tunneling).

those are all Http request methods that can be used ,we will continue with the hyper text transfer protocol at next lesson,goodbye.
Posted by hussam.dh at 4:01 AM 0 comments Links to this post

common http mime types

Type Meaning
application/msword Microsoft Word document
application/octet-stream Unrecognized or binary data
application/pdf Acrobat (.pdf) file
application/postscript PostScript file
application/vnd.ms-excel Excel spreadsheet
application/vnd.ms-powerpoint Powerpoint presentation
application/x-gz Gzip archive
application/x-java-archi JAR file
application/x-java-vm Java bytecode (.class) file
application/zip Zip archive
audio/basic Sound file in .au or .snd
audio/x-aiff AIFF sound file
audio/x-wav Microsoft Windows sound file
audio/midi MIDI sound file
text/css HTML cascading style sheet
text/html HTML document
text/plain Plain text
text/xml XML document
image/gif GIF image
image/jpeg JPEG image
image/png PNG image
image/tiff TIFF image
video/mpeg MPEG video clip
video/quicktime QuickTime video clip
Posted by hussam.dh at 3:19 AM 0 comments Links to this post

Monday, February 21, 2011

Http hypertext transfer protocol

In this tutorial ,I am talking about Http or Hypertext Transfer Protocol taking the RFC 2616 as main reference.
Http is the official protocol used to transfer web pages from web servers.
To make any http request you need to use an Http URL which has the following form:
http_URL = "http:" "//" host [ ":" port ] [ abs_path [ "?" query ]]
this portion is optional but if the port is not defined then it is assumed to be port 80.
example:
http://www.example.com/requested-page.html?username=value1&password=value2
host: www.example.com
port: 80 because it is not defined obviously.
abs_path: request-page.html
query: ?username=value1&password=value2

also the Http request looks like this:

Http_request_method requested_resource Http_version
Http_request_headers
(blank-line)
POST_DATA

Every time you click a link an Http request is made by your web browser,let's request Google main page by typing http://www.google.com in your address bar then press go,the request looks like this:

GET / HTTP/1.1
Host: www.google.com
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.3) Gecko/20100423 Ubuntu/10.04 (lucid) Firefox/3.6.3
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Proxy-Connection: keep-alive

the requested resource in the previous request is / which indicates the index page of the target host.

Http_requested_method: GET
requested_resource: /
Http_version :HTTP/1.1
HTTP_request_headers:
Host: www.google.com
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.3) Gecko/20100423 Ubuntu/10.04 (lucid) Firefox/3.6.3
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Proxy-Connection: keep-alive

POST_DATA: none

to a closer look,you must use a local Http proxy ,I am using Burp suite v1.1,there also paros and webscarab.
To download the latest version of Burp suite:
http://portswigger.net
after downloading it,go to proxy->options->proxy running on port then write your own port.
After configuring Burp proxy you must configure the proxy settings on your own operating system or your browser to :
proxy :
“localhost”
port: your port

now,just try to type any URL in your browser address bar ,press go to see immediately Burp suite window containing the http request you have just sent.
In next tutorial I will explain each word of the Http request,so goodbye now.
Posted by hussam.dh at 3:18 AM 0 comments Links to this post

recursion

Recursion:
recursion is a programming technique,this technique divides the main problem to
smaller sub-problems in order to make the main problem easier to solve (I mean solve the smaller sub-problems then use those solutions to solve the main problem,
this technique is most used in artificial intelligence applications.
Examples:
how to find 5! using recursion.
we have to find 5! but 5!=5*4!,so let's find 4! then make the calculation 5*4! to find 5!
but you can see also that 4! is equal to 4*3!
we have also 3!=3*2!
and 2!=2*1!
1!=1
so let's go up stairs:
2!=2*1!=2*1=2
and 3!=3*2!=3*2=6
and 4!=4*3!=4*6=24
and 5!=5*4!=5*24=120
recursion works like above,dividing the problem to to subproblems then divide the subproblems to subproblems and repeat the division until you reach the simplest form of the problem as above:
we continue dividing the problem to sub-problem and the sub-problem to sub-problem a\until we reach the simplest form which is 1! which equals 1.
I get this simple example,because it is easy to explain but you can make your programs do harder tasks ,there are some problems that need to be solved by recursion,one of them is the problem of finding the shortest way for a traveler.
Another example is Fibonacci :
Fibonacci(0)=Fibonacci(1)=1
Fibonacci(n)=Fibonacci(n-1)+Fibonacci(n-2)
this problem can be solved using recursion:
we want to find the Fibonacci of 5 using recursion:
Fibonacci(5)=Fibonacci(4)+Fibonacci(3)
and:
Fibonacci(4)=Fibonacci(3)+Fibonacci(2)
and:
Fibonacci(3)=Fibonacci(2)+Fibonacci(1)
and:
Fibonacci(2)=Fibonacci(1)+Fibonacci(0)=2
then let's go up in problem:
Fibonacci(3)=Fibonacci(2)+Fibonacci(1)=2+1=3
and go up again:
Fibonacci(4)=Fibonacci(3)+Fibonacci(2)=3+2=5
and again:
Fibonacci(5)=Fibonacci(4)+Fibonacci(3)=5+3=8
which is the solution of this problem.

You may note that in recursion the function calls itself but with simpler parameters,the function keeps calling itself and without a valid stop condition the function will keep calling itself infinitely
so we must determine the stop condition.
Example:
let's have this C function which find the Fibonacci of the passed parameter:
1.int fibo(int n){
2.if(n==0 || n==1)
3.retrun 1;
4.else
5.return fibo(n-1)+fibo(n-2);
6.}
at line 2 we write the stop condition at which the function stops dividing the problem,and at which the function solves the simplest form of the problem.
And if the condition is not true(the problem is not in its simplest form) then it will divide the problem to more sub-problems :
my function does that when it returns fibo(n-1)+fibo(n-2) which are recursive calls with reduced parameters (n-1 and n-2).

This was a simple idea about recursion,but I think it is enough to understand recursion.
Goodbye.
Posted by hussam.dh at 3:11 AM 0 comments Links to this post

Regular expressions

Regular expressions:
regular expressions is a way to represent strings ,this representation can be done according to common characteristics between the represented strings.
The regular expression consists of special characters that represent a set of ASCII characters
for example:
\s represent whitespace character :space,\t ,\n,\r etc..
\S represent a non-whitespace character
\d represent decimal number 0,1,2,3,4,5,6,7,8,9
\D represent any non decimal character
\w represent any alpha-numeric character [A-Z a-z 0-9]
\W represent any non-alpha-numeric character $ % and so on
. :this dot can be used to represent any ASCII character
and of course regular expression can be any normal string “hello” or “foo” or “12hi”....,but I meant
that we can generalize the expression to match a set of strings in stead of one string,suppose that we want to find all hyper links in a web page,can you do this just by identifying one string,absolutely not you need to define a set of strings that must be hyper links but not necessary identical.
Always with those special characters called(meta characters) you may encounter another special characters:
+:the plus means that the preceding character may be repeated in the represented string once or more,here is an example to understand:
the expression “a+” will represent the following strings:
a
aa
aaa
aaaaa
aaaaaaaaaaaaaaaaaaaaaaa
and so on.
*:this means that the preceding character may be repeated zero or more times,example:
the expression 1* will represent the set:
"”//nothing
1
11
11111
….......//of course this means that the character 1 may repeat any number of times
another example:
a1*,this will represent the following:
a
a1
a111
a111111
and so on.
?:indicates that the preceding character is repeated zero or one time,example:
2f? this will represent the following:
2 and 2f
{n} is used to indicate that the preceding characters repeats exactly n times,examples:
p{2}:matches pp
{n,} is used to indicate that the preceding characters repeats n or more times:
p{2,}:matches pp,ppp,ppppppppppp,ppppp,etc..
{n,m} indicates that the preceding character repeats at least n times but not more than m times:
p{2,4} matches:pp,ppp,pppp

we use the [ ] to indicate that any one of the enclosed characters must be in the matched string:
examples:
[bc]:indicates that the matched string must contain at least one character of b and c (I mean b or c)
another ex.:
the regular expression :[bc]at :may match “bat” or “cat” but not “bcat”
the regular expression :[ca]r may match “cr” and “ar” and not “car”

we may use ^ to indicate that not any one of the enclosed characters,example:
[^bc]at :this may match any word of three characters that ends with “at” but this word mustn't
start with b or c.
[^abc]:matches any character except a ,b or c.
you can use [] to identify a range of characters :
[a-z]:means any small letter character from a to z
[a-zA-Z]: means any letter(small or capital).
[a-zA-Z0-9]:means any alpha-numeric character:decimal or letter.

[b-d[r-u]]:any character of the two ranges b-d(b,c,d) or r-u(r,s,t,u).//also written as [[b-d][r-u]]or
[b-dr-u],this is a union of two ranges as above.

Besides the union between ranges there also intersection between them:
[b-d&&[a-z]]:means any character shared between the ranges b-d and a-z which are b-d
[a-z&&[^bc]]:means any character from the range a-z but not any of b and c (I mean [a[d-z]]).
[a-z&&[^m-p]]: means any character of the range a-z and not from the range [m-p](I mean
[a-l[qz]]).
Before taking examples,you must know that each special character such as:
\,.,{,},[,],^,(,),*,+ etc
must be preceded by \ in order to handle it as normal characters:
I mean that we knew what * means in regular expressions ,but when preceded with \ it will match
* and just *
Let's take some examples:
we want to write the regular expression that matches an email address(something@example.com)
first we must identify the characters that make sense that this string is an email address (I mean
how do you know that this string is an email?
Answer:you may recognize any email address by @ and . Which are consistent for all email
addresses.
You may also determine any rules that may be applied to the email form such as
the email address mustn't contain any whitespace character in the something portion,
the something portion must not contain @ character ,
example portion must not contain the dot . or the @ character
The last portion com must not contain the dot  or the @ character
let's assume that those was the main things about email address form and let's write the regular expression:
[^@\s]+@[^@\.\s]+\.[^@\.\s]+

another example,is to write a regular expression to match a hyper link:
<a href=”www.example.com”>www.example.com</a>
you can see that the hyper link always starts with <a and ends with </a> and
after <a you may see some attributes like href and alt but all the stuff between <a and its > must not contain > and the same for the text between the tags which is here www.example.com,so the expression will be:
<a[^>]+>[^<]+</a>
sometimes we can group a string with () to indicate for instance that it may be repeated one or more times :
(hello)+:means that hello string will be repeated one or more times, this will match:
hello
hellohello
hellohellohello ,etc...

but () have another use which is to save the strings which match the enclosed expression to memory.
Saving is done by grouping the enclosed expressions and  numbering the groups from left to right and from outside to inside,as in this example:
<a([^>]+)>([^<]+)</a>
       1            2
you see that we enclosed the anchor text with () , we can get this text by calling group 1(we said from left to right and from outside to inside),because group 0 is the whole expression,the scheme of calling the groups is done according to the programming language being used.
Let's get the real URL that the hyper link links to:
<a[^>]+href=([^>]+)>[^<]+</a>
the value of href is the group 1 matched string.
sometimes we need to know if the matched expression is a single word or if it is at the beginning of the line or at the end of it,this can be done using boundaries(^ $ \b \B etc).
Here are those boundaries:
when the expression starts with ^ this means the expression matches a string only if the matched string is at the beginning of a line:
^how will match  “how” only if it is at the beginning of the line:
let's have:
how how how
then our expression will match only the first how

$ is used after an expression to indicate that it is at the end of the line:
how$ will match “how” If it is at the end of the line.
Let's have:
how how how
then our expression will match only the last how
but what if we have the expression ^how$
then this will match “how” if and only if it is alone on the line,
I mean ,for example,let's have the text:
how are you?
How
good

our expression will match only the how at the second line because It is alone in the line.

\b is a word boundary ,if we want to match the word dog just as a single word:
the expression \bdog\b will match the string dog if and only if it was a single word.
Let's have the string:
the dog name is doggie
our expression (\bdog\b) will match:
dog starting at index 4
but not the string dog starting at index 16 because it is not a single word.

\B is a non-word boundary ,example:
\bdog\B will match any word starting with the three letters dog example:doggie
let's take the previous example:
the dog name is doggie
our expression will match the string “dog” starting at index 16.
goodbye.
Posted by hussam.dh at 3:02 AM 0 comments Links to this post

Sunday, February 13, 2011

follow-me

hi every one,
you can follow me on twitter:
http://www.twitter.com/ilearn_computer
Posted by hussam.dh at 2:38 AM 0 comments Links to this post

Tuesday, December 7, 2010

Address Resolution Protocol

Address Resolution Protocol

ARP and RARP Address Translation

Address Resolution Protocol (ARP) provides a completely different function to the network than Reverse Address Resolution Protocol (RARP). ARP is used to resolve the ethernet address of a NIC from an IP address in order to construct an ethernet packet around an IP data packet. This must happen in order to send any data across the network. Reverse address resolution protocol (RARP) is used for diskless computers to determine their IP address using the network.

 

Address Resolution Protocol (ARP):

In an earlier section, there was an example where a chat program was written to communicate between two servers. To send data, the user (Tom) would type text into a dialog box, hit send and the following happened: 

  1. The program passed Tom's typed text in a buffer, to the socket.
  2. The data was put inside a TCP data packet with a TCP header added to the data. This header contained a source and destination port number along with some other information and a checksum.
  3. The TCP packet was be placed inside an IP data packet with a source and destination IP address along with some other data for network management.
  4. The IP data packet was placed inside an ethernet data packet. This data packet includes the destination and source address of the network interface cards (NIC) on the two computers. The address here is the hardware address of the respective cards and is called the MAC address.
  5. The ethernet packet was transmitted over the network line.
  6. With a direct connection between the two computers, the network interface card on the intended machine, recognized its address and grabbed the data.
  7. The IP data packet was extracted from the ethernet data packet.
  8. The TCP data packet was extracted from the IP data packet.
  9. The data was extracted from the TCP packet and the program displayed the retrieved data (text) in the text display window for the intended recipient to read. 
In step 4 above, the IP data was going to be placed inside an ethernet data packet, but the computer constructing the packet does not have the ethernet address of the recipient's computer. The computer that is sending the data, in order to create the ethernet part of the packet, must get the ethernet hardware (MAC) address of the computer with the intended IP address. This must be accomplished before the ethernet packet can be constructed. The ethernet device driver software on the receiving computer is not programmed to look at IP addresses encased in the ethernet packet. If it did, the protocols could not be independent and changes to one would affect the other. This is where address resolution protocol (ARP) is used. Tom's computer sends a network broadcast asking the computer that has the recipient's IP address to send it's ethernet address. This is done by broadcasting. The ethernet destination is set with all bits on so all ethernet cards on the network will receive the data packet. The ARP message consists of an ethernet header and ARP packet. The ethernet header contains:
  1. A 6 byte ethernet destination address.
  2. A 6 byte ethernet source address.
  3. A 2 byte frame type. The frame type is 0806 hexadecimal for ARP and 8035 for RARP
The encapsulated ARP data packet contains the following: 
  1. Type of hardware address (2 bytes). 1=ethernet.
  2. Type of protocol address being mapped( 2 bytes). 0800H (hexadecimal) = IP address.
  3. Byte size of the hardware address (1 byte). 6
  4. Byte size of the protocol address (1 byte). 4
  5. Type of operation. 1 = ARP request, 2=ARP reply, 3=RARP request, 4=RARP reply.
  6. The sender's ethernet address (6 bytes)
  7. The sender's IP address (4 bytes)
  8. The recipient's ethernet address (6 bytes)
  9. The recipient's IP address (4 bytes) 
When the ARP reply is sent, the recipient's ethernet address is left blank. 

In order to increase the efficiency of the network and not tie up bandwidth doing ARP broadcasting, each computer keeps a table of IP addresses and matching ethernet addresses in memory. This is called ARP cache. Before sending a broadcast, the sending computer will check to see if the information is in it's ARP cache. If it is it will complete the ethernet data packet without an ARP broadcast. Each entry normally lasts 20 minutes after it is created. RFC 1122 specifies that it should be possible to configure the ARP cache timeout value on the host. To examine the cache on a Windows, UNIX, or Linux computer type "arp -a".

If the receiving host is on another network, the sending computer will go through its route table and determine the correct router (A router should be between two or more networks) to send to, and it will substitute the ethernet address of the router in the ethernet message. The encased IP address will still have the intended IP address. When the router gets the message, it looks at the IP data to tell where to send the data next. If the recipient is on a network the router is connected to, it will do the ARP resolution either using it's ARP buffer cache or broadcasting. 

Reverse Address Resolution Protocol (RARP):

As mentioned earlier, reverse address resolution protocol (RARP) is used for diskless computers to determine their IP address using the network. The RARP message format is very similar to the ARP format. When the booting computer sends the broadcast ARP request, it places its own hardware address in both the sending and receiving fields in the encapsulated ARP data packet. The RARP server will fill in the correct sending and receiving IP addresses in its response to the message. This way the booting computer will know its IP address when it gets the message from the RARP server.


thanks for reafing.


note:reference:www.comptechdoc.com






 

Posted by hussam.dh at 9:07 AM 0 comments Links to this post
Subscribe to: Posts (Atom)