You've probably heard of Bing, one of the largest search engines around, love it or loathe it. But did you know it has a nifty API? Bing Search APIs have moved recently out of Cognitive Services (as of October 30, 2020).
These APIs include local business search, news, and image search. Wouldn't it be cool if we could use Bing News Search to quickly and easily find out what's going on in the world? Maybe you want to quickly check up on a coffee break, remove the visuals from your pandemic doom-scrolling, or you need inspiration for a tweet!
Today, I'm going to show you how to use Python to create a Command Line Interface (CLI) to search for trending articles, phrases you're particularly interested in, or just see what's happening in the world of business.
What you'll need
- An Azure tenant (create a free account if you don't have one)
- Basic Python Proficiency
- A local installation of Git
- A local installation of your preferred Python distribution
- Visual Studio Code (or similar code editor)
I'll walk you through the process of setting up a Bing Search API including the News feature, and then walk you through the application.
Creating your Bing News Resource
Use this link to jump directly to the Bing Search marketplace template, or search for Bing Search v7 in the marketplace.
However you find this, creating the resource will need you to be familiar with the pricing tier. Not all of the tiers include Bing News Search.
For the purposes of this exercise, I've used the 'Free' instance, which allows up to 1,000 free transactions per month at the time of writing - but if you intend to use the tool often, you'll want to look at the paid tiers.
Once you've read the privacy statement, filled in the form, and hit create, the resource will deploy. Time to get a coffee!
Getting your Bing News Endpoint and Keys
When your resource is deployed, open it in the Azure Portal, and look at the Overview screen.
Take a note of the Endpoint, as you'll need this URL for the application. My endpoint isn't part of a multi-service subscription, so it doesn't have the region in the URL, so it's just
Click on either 'Keys and Endpoint' or the 'Click here to manage keys' link to open the 'Keys and Endpoint' tab.
Keep hold of the endpoint from earlier, and click on the blue copy icon to copy the API key, and keep this safe as well.
Configuring the Application
Fire up a terminal, or open up an integration terminal in your preferred code editor - I'm a fan of Visual Studio Code - and clone the Git repository for the project.
Make sure you've changed your current working directory, to wherever you want the code to go, and then run the clone command.
git clone https://github.com/Sealjay-clj/bingnews-cli.git
.env.template file in the
bingnews-cli/ directory to
.env, replacing the content there with the Bing Search Key, and the Bing Search Endpoint you noted down earlier.
Install the Python Dependencies
Rather than installing all of the dependencies into the global environment, we'll create a separate environment.
To create a separate Python environment for your installation, and activate it, there are two common options.
I prefer to use miniconda, but you can use a
venv if you prefer.
a. Using a Conda distribution
If you are using a distribution of conda, you may want to create a new conda environment, rather than use venv:
```bash $ conda create --name bingnews python=3.9 -y $ conda activate bingnews $ pip3 install -r requirements-dev.txt ```
b. Using a Python virtual environment
On Debian Linux distributions, you may need to run
sudo apt-get install python3-venv before these instructions. On other distributions, I don't have a clue!
$ python3 -m venv env $ source env/bin/activate $ pip3 install -r requirements-dev.txt
On Windows, you may need to use
pip commands where there are references to the
At this point, you should have a configured Python environment, and we're just about ready to try this out!
cd bingnews-cli into the same directory as
bingnews.py, and run
Hopefully, you're seeing a screen along these lines!
How does it work?
The application is surprisingly simple. Most of the intelligence takes place in Bing Search, and then Click is used to generate a CLI. Click is a Python package focussed on creating command line interfaces.
The command line has three options:
- Search by Category. Example:
python3 bingnews.py cat -c Business
- Search by Phrase. Example:
python3 bingnews.py phrase -p "Future of Computing"
- Search for Trending articles. Example:
python3 bingnews.py trend
The basic request
The basic request is made up of the
search_and_output_bing function, which is called for all of the various API queries.
This queries the Bing News endpoint, and then requests that the output is printed with
print_bing_results, which in turn formats the output for presentation in
clean_bing_article_list, applying some tweaks for the varied result-set returned by trending articles, in
Composing functions with Click
In order to make functions available on the command-line,
click allows us to use decorators to automatically generate help information for the command line, pass arguments to the function, and provide prompts if information is not provided by the user. Some defaults are also provided which can be overridden on the command-line, such as defaulting to the
Requests are cached for five minutes, in case the user runs the same search, or in case the shortener is provided similar news article URLs, to prevent requesting the same URL twice for shortening from an endpoint.
Tabulate is used to render the dictionary output from the API, and make it easily readable in the terminal.
URLs provided by Bing News are shortened with the
pyshorteners package,and the qpsru service, to ensure they don't wrap off the terminal, and break the link.
What packages does this use
The key packages are as follows:
|click||Handles the CLI functionality|
|tabulate||Formats the results as a table|
|python-dotenv||Retrieves the configuration values from the .env file|
|requests||Handles the API request|
|requests-cache||Caches the requests if we make the same query, or shorten the same URL (pyshorteners is based on the requests package as well, and the qspru shortener uses GET requests)|
|pyshorteners||Shortens the URls - I use psru, as this shortener doesn't require an API key|
If you were to use this frequently in production, you'd want to add tests, and install this locally with a
setup.py file. But for now, it's a good start for a CLI that can be re-used - and I'll be extending the functionality in a future post.
Thanks for reading, and I hope you found this interesting!