Python Simple Web Crawler

A web crawler (also known as a web spider or web robot) is a program or automated script which browses the World Wide Web in a methodical, automated manner and this process is called Web crawling or spidering.

Simple Program For Web Crawler...

import urllib2

from urllib2 import urlopen

website = 'https://google.com'

sourceCode = urllib2.urlopen(website).read()

print sourceCode

This Program will capture the Source Code of the given URL and print on the Python Shell. In this way, You extract the information from the website. In next post, we will discuss about that.

Basic Python Program

Basic Program

Python Variable Types

 Variables are nothing but reserved memory locations to store values. i.e when you create a variable you reserve some space in memory.

 Assigning Values to Variables

The operand to the left of the = operator is the name of the variable and the operand to the right of the = operator is the value stored in the variable. For example:

counter = 100           # An integer assignment
miles   = 1000.0        # A floating point
name    = "John"       # A string

print counter
print miles
print name

Multiple Assignment:

Python allows you to assign a single value to several variables simultaneously. For example:

a = b = c = 1

a, b, c = 1, 2, "john"

Standard Data Types:

The data stored in memory can be of many types. For example, a person's age is stored as a numeric value and his or her address is stored as alphanumeric characters. Python has various standard types that are used to define the operations possible on them and the storage method for each of them.

Python has five standard data types:

• Numbers
  
  var1 = 1
  var2 = 10
   
• String
   str = 'Hello World!'
  
  print str                    # Prints complete string
  print str[0]              # Prints first character of the string
  print str[2:5]          # Prints characters starting from 3rd to 5th
  print str[2:]            # Prints string starting from 3rd character
  print str * 2           # Prints string two times
  print str + "TEST" # Prints concatenated string
  
  This will produce the following result:
   
  
  Hello World!
  H
  llo
  llo World!
  Hello World!Hello World!
  Hello World!TEST
  
  
• List
   list = [ 'abcd', 786 , 2.23, 'john', 70.2 ]
  tinylist = [123, 'john']
  
  print list                  # Prints complete list
  print list[0]             # Prints first element of the list
  print list[1:3]         # Prints elements starting from 2nd till 3rd
  print list[2:]           # Prints elements starting from 3rd element
  print tinylist * 2    # Prints list two times
  print list + tinylist # Prints concatenated lists
  
  This will produce the following result:
  
  
  
  ['abcd', 786, 2.23, 'john', 70.200000000000003]
  abcd
  [786, 2.23]
  [2.23, 'john', 70.200000000000003]
  [123, 'john', 123, 'john']
  ['abcd', 786, 2.23, 'john', 70.200000000000003, 123, 'john'] 
• Tuple
• A tuple is another sequence data type that is similar to the list. A tuple consists of a number of values separated by commas. Unlike lists, however, tuples are enclosed within parentheses.
• The main differences between lists and tuples are: Lists are enclosed in brackets ( [ ] ) and their elements and size can be changed, while tuples are enclosed in parentheses ( ( ) ) and cannot be updated. Tuples can be thought of as read-only lists. For example:
  
  tuple = ( 'abcd', 786 , 2.23, 'john', 70.2  )
  tinytuple = (123, 'john')
  
  print tuple           # Prints complete list
  print tuple[0]        # Prints first element of the list
  print tuple[1:3]      # Prints elements starting from 2nd till 3rd
  print tuple[2:]       # Prints elements starting from 3rd element
  print tinytuple * 2   # Prints list two times
  print tuple + tinytuple # Prints concatenated lists
  
  This will produce the following result:
  
  
  ('abcd', 786, 2.23, 'john', 70.200000000000003)
  abcd
  (786, 2.23)
  (2.23, 'john', 70.200000000000003)
  (123, 'john', 123, 'john')
  ('abcd', 786, 2.23, 'john', 70.200000000000003, 123, 'john')

• Dictionary
• Python's dictionaries are kind of hash table type. They work like associative arrays or hashes found in Perl and consist of key-value pairs. A dictionary key can be almost any Python type, but are usually numbers or strings. Values, on the other hand, can be any arbitrary Python object.

          dict = {}
          dict['one'] = "This is one"
          dict[2]     = "This is two"

         tinydict = {'name': 'john','code':6734, 'dept': 'sales'}


        print dict['one']             # Prints value for 'one' key
        print dict[2]                   # Prints value for 2 key
        print tinydict                 # Prints complete dictionary
        print tinydict.keys()     # Prints all the keys
        print tinydict.values() # Prints all the values

       This will produce the following result:
   
               This is one
               This is two
               {'dept': 'sales', 'code': 6734, 'name': 'john'}
               ['dept', 'code', 'name']
               ['sales', 6734, 'john'] 

Python 2.7 Tutorial By Techno-Class

Python 2.7

Introduction

 As you know, python is a programming language, easy to implement and a great language for the beginner programmers. It has simple but effective approach to object-oriented programming as well as efficient high-level data structures. It is an ideal language for scripting web crawler, natural language tool kit program and application development in many areas on most platforms.

Python Features:

• Easy to learn 
• Easy to read  
• Easy understanding
• Bulk standard library 
• Portable on variety of platforms
• provides interfaces to all commercial databases
• Supports GUI applications

 

Installation process 

 Download python 2.7 from official website https://www.python.org/downloads/  and install the software.            

         

  After the installation of software we will start simple programming.First we start Python command line or IDLE Python GUI.

Type on command line 2+2 and press Enter, that gives the result.

 

Hello World Program 

>>> print 'Hello World !!' 

Hello World !!

You can also try this sample program to understand the basics. 

      >>> a = 5

>>> print a
5
>>> name = 'Rahul'
>>> print name
Rahul
>>>