Q: How to get Google like url excerpts in python I have written a code like this:
#pip3 install google 
from googlesearch import search

query = 'java'


for i in search(query,        # The query you want to run
                tld = 'com',  # The top level domain
                lang = 'en',  # The language
                num = 10,     # Number of results per page
                start = 0,    # First result to retrieve
                stop = None,  # Last result to retrieve
                pause = 0,  # Lapse between HTTP requests
                safe = 'high'

               ):
    print(i)

in the above, I am simply getting the url link. How can I get google like excerpts for each url. Like the attached

A: I don't think it is possible to do it with the package BUT if you use the requests module on python with some web scraping libraries (beautifulsoup) you can get those descriptions with the HTML tag (<meta name="description" content="google description">)

A: You can scrape it using requests, bs4 library and user-agent.
Make sure you're using user-agent, because if you don't, your script would have a default user-agent (it could be a tablet or phone) which will show different classes and because of it you will get an empty output.
Here's a code and replit.com(java search result on repl.it):
from bs4 import BeautifulSoup
import requests
import lxml
import json

headers = {
  'User-agent':
  "Mozilla/5.0 (Windows NT 10.0; Win6a4; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36 Edge/18.19582"
}

html = requests.get('https://www.google.com/search?hl=en-US&q=best lasagna recipe ever', headers=headers).text

soup = BeautifulSoup(html, 'lxml'))

summary = []

for container in soup.findAll('div', class_='tF2Cxc'):
  heading = container.find('h3', class_='LC20lb DKV0Md').text
  article_summary = container.find('span', class_='aCOpRe').text
  link = container.find('a')['href']

summary.append({
  'Heading': heading,
  'Article Summary': article_summary,
  'Link': link,
})

print(json.dumps(summary, indent=2, ensure_ascii=False))

JSON output:
    [
  {
    "Heading": "World's Best Lasagna Recipe - Allrecipes.com",
    "Article Summary": "Ingredients. 1 pound sweet Italian sausage. ¾ pound lean ground beef. ½ cup minced onion. 2 cloves garlic, crushed. 1 (28 ounce) can crushed tomatoes. 2 (6 ounce) cans tomato paste. 2 (6.5 ounce) cans canned tomato sauce. ½ cup water.",
    "Link": "https://www.allrecipes.com/recipe/23600/worlds-best-lasagna/"
  },
  {
    "Heading": "The BEST Lasagna Recipe Ever! | The Recipe Critic",
    "Article Summary": "Dec 22, 2019 — The BEST Classic Lasagna Ever has layers of sautéed ground beef and Italian sausage that are cooked together, sweet tomato sauce, Italian ...",
    "Link": "https://therecipecritic.com/lasagna-recipe/"
  },
  {
    "Heading": "The Most Amazing Lasagna Recipe - The Stay At Home Chef",
    "Article Summary": "The Most Amazing Lasagna Recipe is the best recipe for homemade Italian-style lasagna. The balance ... This recipe is so good—it makes the kind of lasagna people write home about! ... Hands down absolutely the best lasagna recipe ever!",
    "Link": "https://thestayathomechef.com/amazing-lasagna-recipe/"
  },
  {
    "Heading": "Best Lasagna - Cafe Delites",
    "Article Summary": "My mama's famous lasagna recipe is hands down the best lasagna I have ever had in my life. She learnt her ways from her Italian friends when she lived in New ...",
    "Link": "https://cafedelites.com/best-lasagna/"
  },
  {
    "Heading": "The Best Lasagna Recipe EVER | Fail Proof Recipe | Lauren's ...",
    "Article Summary": "Start with a bit of meat sauce in the bottom of a large casserole dish or a plain 9×13 and line the bottom with pasta. Top with the cheese mixture and meat sauce.",
    "Link": "https://laurenslatest.com/best-lasagna-recipe/"
  },
  {
    "Heading": "Best Lasagna Recipe: How to Make It | Taste of Home",
    "Article Summary": "Want to know how to make lasagna for a casual holiday meal? You can't go wrong with this deliciously rich meat lasagna recipe. ... I made this lasagna for my fiance, he said this lasagna was the best he ever tasted, I will never buy frozen ...",
    "Link": "https://www.tasteofhome.com/recipes/best-lasagna/"
  },
  {
    "Heading": "Easy Homemade Lasagna {Classic Dinner!} - Spend With ...",
    "Article Summary": "May 19, 2020 — noodles – sauce (bake) – cheese. Spread about a cup of meat sauce into a 9×13 pan. Add a layer of noodles. Top the noodles with some of ...",
    "Link": "https://www.spendwithpennies.com/easy-homemade-lasagna/"
  },
  {
    "Heading": "The Best Lasagna Recipe {Simple & Classic} - Simply Recipes",
    "Article Summary": "Feb 19, 2019 — Ingredients · 1 pound lean ground beef (chuck); 1/2 onion, diced (about 3/4 cup); 1/2 large bell pepper (green, red, or yellow), diced (about 3/4 cup) ...",
    "Link": "https://www.simplyrecipes.com/recipes/lasagna/"
  },
  {
    "Heading": "Best Lasagna Recipe - How to Make Lasagna From Scratch",
    "Article Summary": "Dec 15, 2020 — The Best Lasagna. Ever. · Bring a large pot of water to a boil. · Meanwhile, in a large skillet or saucepan, combine ground beef, sausage, and garlic ...",
    "Link": "https://www.thepioneerwoman.com/food-cooking/recipes/a11728/best-lasagna-recipe/"
  }
]

Alternatively, you can use Google Search Engine Results API from SerpApi.
Part of JSON:
"organic_results": [
  {
    "position": 1,
    "title": "World's Best Lasagna Recipe - Allrecipes.com",
    "link": "https://www.allrecipes.com/recipe/23600/worlds-best-lasagna/",
    "displayed_link": "https://www.allrecipes.com › ... › European › Italian",
  }
]

Code to integrate:
import os
from serpapi import GoogleSearch

params = {
    "engine": "google",
    "q": "best lasagna recipe ever",
    "api_key": os.getenv("API_KEY"),
}

search = GoogleSearch(params)
results = search.get_dict()

for result in results["organic_results"]:
   print(f"Title: {result['title']}\nLink: {result['link']}")

Output:
Title: World's Best Lasagna Recipe - Allrecipes.com, 
Link: https://www.allrecipes.com/recipe/23600/worlds-best-lasagna/
Title: Best Lasagna - Cafe Delites, 
Link: https://cafedelites.com/best-lasagna/
Title: The Most Amazing Lasagna Recipe - The Stay At Home Chef, 
Link: https://thestayathomechef.com/amazing-lasagna-recipe/
Title: The BEST Lasagna Recipe Ever! | The Recipe Critic, 
Link: https://therecipecritic.com/lasagna-recipe/
Title: The Best Lasagna Recipe EVER | Fail Proof Recipe | Lauren's ..., 
Link: https://laurenslatest.com/best-lasagna-recipe/
Title: Best Lasagna Recipe - How to Make Lasagna From Scratch, 
Link: https://www.thepioneerwoman.com/food-cooking/recipes/a11728/best-lasagna-recipe/
Title: Best Lasagna Recipe: How to Make It | Taste of Home, 
Link: https://www.tasteofhome.com/recipes/best-lasagna/
Title: Easy Homemade Lasagna {Classic Dinner!} - Spend With ..., 
Link: https://www.spendwithpennies.com/easy-homemade-lasagna/
Title: The Best Lasagna Recipe {Simple & Classic} - Simply Recipes, 
Link: https://www.simplyrecipes.com/recipes/lasagna/


Disclaimer: I work for SerpApi.

