Yes Bank withdrawal limit capped at Rs 50000

Yes Bank withdrawal limit capped at Rs 50000

MUMBAI: The Reserve Bank of India (RBI) superseded the board of Yes BankNSE -58.56 % and imposed a month-long moratorium, it said in an announcement late on Thursday. It expects to arrive at a credible restructuring plan in the next few days.

“The Reserve Bank assures the depositors of the bank that their interests will be fully protected and there is no need to panic,” it said in a statement. This is the first time that the central bank has taken such drastic action with respect to a big bank since July 2004 when the regulator got state-run Oriental Bank of Commerce (OBC) to take over Global Trust Bank to rescue the private sector lender.

The RBI action follows the lender’s inability to raise funds that would have helped it provide against loan losses. Prashant Kumar, former deputy managing director at State Bank of IndiaNSE -6.46 % (SBI), will be the administrator of Yes Bank, RBI said.

Depositors will be restricted to a maximum withdrawal of Rs 50,000 even if they have multiple accounts, a government gazette notification said. RBI will relax the withdrawal limit in the event of medical emergencies, higher education fees or marriage expenses — up to a cap of Rs 5 lakh. Drafts and pay orders issued so far will be paid in full, it said

RBI Blames Lax Governance Standards

“In the absence of a credible revival plan, and in public interest and the interest of the bank’s depositors, (the RBI) had no alternative but to apply to the central government for imposing a moratorium,” the central bank said. “The Reserve Bank will explore and draw up a scheme in the next few days for the bank’s reconstruction or amalgamation and with the approval of the central government, put the same in place well before the period of moratorium of 30 days ends so that the depositors are not put to hardship for a long period of time.”


The central bank said its preferred option had been a market-related solution.

“Since a bank and market-led revival is a preferred option over a regulatory restructuring, the Reserve Bank made all efforts to facilitate such a process and gave adequate opportunity to the bank’s management to draw up a credible revival plan, which did not materialise,” the RBI said. “In the meantime, the bank was facing regular outflow of liquidity.’’ The central bank blamed lax governance standards at Yes bank over the past few years.

“The financial position of Yes Bank Ltd has undergone a steady decline largely due to inability of the bank to raise capital to address potential loan losses and resultant downgrades, triggering invocation of bond covenants by investors, and withdrawal of deposits,” said the RBI. “The bank has also experienced serious governance issues and practices in the recent years which have led to steady decline of the bank.”

Yes Bank will be allowed to repay loans or advances granted against government securities or other securities to the bank by the RBI or by any other bank and remaining unpaid as of Thursday. It would also be allowed to operate its account with the RBI.

The lender, which has seen its fortunes slide over the past 18 months, has been in talks with equity investors over the past year, but had failed to come up with a concrete investment plan.
“The bank management had indicated to the Reserve Bank that it was in talks with various investors and they were likely to be successful,’’ said the RBI statement. “The bank was also engaged with a few private equity firms for exploring opportunities to infuse capital. These investors did hold discussions with senior officials of the Reserve Bank but for various reasons eventually did not infuse any capital.”

Last month, Yes Bank had informed exchanges that about half-a-dozen investors, including JC Flowers & Co., Tilden Park Capital Management, OHA (UK), part of Oak Hill Advisors, and Silver Point Capital, had submitted “nonbinding” expressions of interest. Yes Bank had also postponed its December quarter results announcement, citing fundraising talks, to March 14.

The lender turned wobbly due to surging bad loans and management uncertainty when the Reserve Bank of India declined to extend the term of founder Rana Kapoor as chief executive in 2018. Under his successor Ravneet Gill, the bank managed to raise one round of funds through share sales to institutional investors, but that didn’t prove to be enough.

yes bank news

Shares in India's Yes Bank plunge over 70% on withdrawal limits

Yes Bank has been struggling under a mountain of bad loans
Shares in India's fourth-largest private lender Yes Bank plunged more than 70 percent on Friday after the central bank seized control and imposed withdrawal limits.

Queues formed outside Yes Bank branches after the announcement late Thursday that customers can only withdraw 50,000 rupees ($678) over the next 30 days.
India has been grappling with a liquidity crunch caused by the near-collapse more than a year ago of IL&FS, one of the nation's biggest shadow banks -- finance houses responsible for significant consumer lending.
Yes Bank has been particularly badly hit as it struggles under a mountain of bad loans.
Its weakened position was "largely due to inability of the bank to raise capital to address potential loan losses and resultant downgrades", the Reserve Bank of India, said Thursday.
"The bank has also experienced serious governance issues and practices in the recent years, which have led to a steady decline of the bank," it added but said there was "no need to panic".
"It's not clear at the moment and that is making some people panic a bit. Even I am at the moment," Devika, a student and Yes Bank account holder, told AFP as she queued to withdraw money in New Delhi.
By midday in Mumbai (0630 GMT), Yes Bank shares had recovered slightly but were still down 61 percent at 14.30 rupees.

Rupee, India Stocks Tumble After RBI Takes Control of Yes Bank

India’s rupee weakened toward a record low and stocks tumbled after the central bank seized control of beleaguered Yes Bank Ltd., intensifying the risk-off mood fueled by the spread of coronavirus cases in India.

The S&P BSE Sensex slid 3% to 37,322.59 in Mumbai at 10:35 a.m., entering correction zone. The rupee declined as much as 1.1% and was set for a record low of 74.4825 per dollar seen in 2018.

“Investors have turned risk averse and the contagion is likely to hit smaller bank and non-bank finance companies,” said Abhimanyu Sofat, Mumbai-based head of research at IIFL Securities Ltd. “How soon the RBI finalizes the rescue plan is key as persistent operational curbs increase uncertainty.”

Read:Contagion Risk on India Seen From Modi’s Move to Seize Yes Bank

The Reserve Bank of India put strict limits on the lender’s operations while a rescue plan is devised. Under a government-backed proposal, State Bank of India, the nation’s largest lender, will lead a group that will inject new capital into Yes Bank, people familiar with the matter said on Thursday.

The restrictions could hardly have come at a worse time, with Indian assets already reeling from the economic impact of the outbreak. Primary schools in the capital New Delhi have been closed until March 31 to prevent the spread of the virus with the number of confirmed patients across the country jumping to 29 this week.

Read: India Seizes Embattled Yes Bank After Capital Raising Plans Fail

The RBI capped Yes Bank withdrawals at 50,000 rupees ($682) and ordered it not to issue new loans. The lender’s shares plunged 40% and State Bank fell 7%. Yield on 10-year government debt was down 9 basis points to 6.15%.

Friday’s drop is likely to cap a third straight week of losses for the rupee. While the rupee was initially insulated from the drop in Asian currencies, a rising number of virus case detections in India has led to worries about the potential fallout on economic activity.

“INR is emerging as the underperformer in the region today, likely due to the relative underperformance of India equities,” said Dushyant Padmanabhan, strategist at Nomura Holdings Inc.

The runaway winner has been the country’s sovereign bonds. Helped by the central bank’s unconventional policies like infusing long-term cheap money, yields have plummeted, particularly on the shorter end of the yield curve.

Yes Bank's Bonfire of Insanity Was Left to Burn

Yes Bank Ltd. has become no bank.

Late Thursday evening in Mumbai, India placed the troubled private-sector lender under moratorium. Depositors will only be allowed to withdraw the rupee equivalent of less than $700 for the next month and the bank won’t be allowed to make new loans. In the meantime, the central bank will seek a way to revamp or merge a bank whose previous owner-manager has left it with few good assets and a truck load of liabilities. In September, the bank had total deposits of 2.1 trillion rupees ($28 billion). Yes delayed its December-quarter results, but assuming that 20% of deposits have since scampered off to safer lenders, authorities still have a $20 billion-plus hole to fill.

Earlier during the day, Bloomberg News reported that the government had approved a bailout by a consortium led by State Bank of India. Why did it wake up so late? It was blindingly obvious even two months ago that the authorities needed to end this theater of the absurd and force SBI to get into the driver’s seat at Yes. It was the only way to stop a run on deposits that could easily become a contagion and knock the wind out of Indian finance.

Yet the authorities dragged their feet, allowing themselves to be fooled by stories of how large private investors were lining up to recapitalize Yes. In the end, nobody came — not Microsoft Corp., not the mysterious Canadian lumber tycoon plotting a takeover from his motel room, not New York-based Cerberus Capital Management LP. Nobody came except SBI, together perhaps with the Life Insurance Corp. of India. They’re only here because the country’s largest bank and biggest life insurer have no option except to do what the government asks of them. They’re the national team.

What should they be paying to play? Nothing, actually. As Macquarie Research says, a bulk of Yes Bank’s junk-rated corporate borrowers are likely to default, as could some of the currently investment-grade debtors. The lender’s net worth of roughly 250 billion rupees should then work out to zero. Yet for the SBI-led alliance to give itself new shares at even 1 or 2 rupees apiece when the stock closed Thursday at nearly 37 rupees will require approval from the securities regulator.

While that shouldn’t pose a problem in a state-sponsored rescue, this isn’t a free lunch for SBI’s shareholders. Their arms are being twisted because the government doesn’t want to explicitly socialize private losses. The pain will linger because there’s no telling who SBI will be asked to rescue next. With lax regulatory supervision and little accountability, there will be more swashbuckling risk takers like Rana Kapoor, the former Yes chief executive officer who ran the bank aground, sold his shares and left a gigantic mess for others to clean up. Similar questions about governance and independence will arise when the government tries to take state-run Life Insurance Corp. public this year or next in a mammoth Saudi Aramco-style IPO.

How long can such Band-Aids work against deep wounds that require direct surgery? There’s no end in sight for India’s multi-year corporate and banking stress. Arindam Som, an analyst at India Ratings and Research, recently estimated that 10.5 trillion rupees, or 16% of the India’s outstanding corporate debt, is vulnerable to default over the next three years. If a quarter of it eventually sours, there will be 2.5 trillion rupees in loan losses. Some of this stress is provided for, but incremental credit costs for lenders could still be in the ballpark of 1.4 trillion rupees — more if India’s economic slowdown gets worse. Where will this money come from, if not from the government?

Even if the embargo on deposit withdrawals gets lifted within a month, some damage is already done. Funds such as Nippon India Life Asset Management Ltd. and Franklin Templeton Asset Management India are exposed to Yes Bank’s toxic debt to the tune of hundreds of million of dollars. The shocks to confidence will reverberate in the shape of heightened risk aversion and stiffer borrowing costs, just as they did after infrastructure financier IL&FS Group went bankrupt in September 2018. Even in that instance, the government stepped in to sell assets and repay borrowers. But the pace of resolution of what turned out to be India’s mini-Lehman moment has been excruciatingly slow.

India doesn’t have a safe way to let even large non-deposit-taking financial firms fail, which is why the authorities needed to handle trouble at a large deposit-taking institution like Yes with more urgency. It was in May last year that the Reserve Bank of India, the banking regulator, used its powers to nominate a director to the bank’s board. Yet, in the end, it couldn’t engineer a better outcome for small savers. By allowing the fire at Yes to trap depositors, even temporarily, the Indian central bank has singed its own reputation.

Coronavirus: How to protect yourself

The Covid-19 coronavirus outbreak is a new illness and scientists are still assessing how it spreads from person to person, but similar viruses tend to spread via cough and sneeze droplets.

When an infected person coughs or sneezes, they release droplets of saliva or mucus. These droplets can fall on people in the vicinity and can be either directly inhaled or picked up on the hands then transferred when someone touches their face, causing infection. For flu, some hospital guidelines define exposure as being within six feet of an infected person who sneezes or coughs for 10 minutes or longer.

1. Wash your hands: wet your hands with clean, running water. Apply soap. Lather your hands, including the backs, between your fingers, and under your nails. Scrub for at least 20 seconds Rinse.

2. Cover your mouth and nose with a tissue when you cough or sneeze, then throw the tissue in a bin and wash hands. If you don’t have a tissue to hand, cough or sneeze into your elbow rather than your hands.

3. Face masks offer some protection as they block liquid droplets. However, they do not block smaller aerosol particles that can pass through the material of the mask.

4.Seek early medical help by phone if you have a fever, cough and difficulty breathing, and share your travel history with healthcare providers.

5. If you have returned from an affected area in China, Iran, South Korea or Italy in the last two weeks, stay indoors and avoid contact with other people for 14 days. This means not going to work, school or public areas.

6. If you have returned from an infected area and develop a high temperature, cough, runny nose, sore throat or difficulty breathing do not leave your home until you have been given advice by a doctor.

7. If visiting live markets in affected areas avoid direct, unprotected contact with live animals and surfaces that have been in contact with animals.

What are the symptoms of the new coronavirus?

What are the symptoms of the new coronavirus?

Here's how you could catch COVID-19.

The novel coronavirus that causes the disease COVID-19 has spread from the site of the original outbreak in China to affect 75 countries around the world. If effective controls aren't put into place, COIVID-19 could ultimately infect between 40% and 70% of the population worldwide in the coming year, according to Harvard T.H. Chan School of Public Health epidemiologist Marc Lipsitch. 

Most of those cases would be mild, and some people might show no symptoms at all. But the prospect of being infected with a new virus can be frightening. The symptoms to look out for, according to the Centers for Disease Control and Prevention (CDC), are fever, coughing and shortness of breath. These symptoms usually appear between two days and two weeks of exposure to the virus. 

According to a report in the Journal of the American Medical Association, as many as 98% of COVID-19 patients have a fever, between 76% and 82% have a dry cough, and 11% to 44% report exhaustion and fatigue. 

The disease appears to become more severe with age, with the 30- to 79-year-old age range predominating the detected cases in Wuhan, where the outbreak began, according to a study in JAMA. Children seem to be at less risk of suffering noticeable symptoms of the disease. 

In more serious cases of COVID-19, patients experience pneumonia, which means their lungs begin to fill with pockets of pus or fluid. This leads to intense shortness of breath and painful coughing. 

Currently, testing for the virus that causes COVID-19 in the United States is limited to people with severe symptoms, according to Paul Biddinger, the director of the emergency preparedness research, evaluation and practice program at the Harvard T.H. Chan School of Public Health, who spoke in a university webcast March 2. This means that it isn't appropriate to be tested at the first sign of a fever or sniffle. Seeking medical care for mild illness can also potentially transmit that illness, or lead to catching new illnesses in the hospital or clinic, Biddinger added. 

If you become ill with these symptoms and live in or have traveled to an area where COVID-19 is spreading, which now includes parts of the U.S., the CDC recommends calling your doctor first rather than traveling to a clinic. Physicians work with state health departments and the CDC to determine who should be tested for the new virus. However, the CDC also recommends that people with COVI-19 or any respiratory illness monitor their symptoms carefully. Worsening shortness of breath is reason to seek medical care, particularly for older individuals or people with underlying health conditions. The CDC information page has more information on what to do if you are sick. 

Coronavirus basics

The novel coronavirus, now called SARS-CoV-2, causes the disease COVID-19. The virus was first identified in Wuhan, China, on Dec. 31, 2019. Since then, it has spread to every continent except Antarctica. The death rate appears to be higher than that of the seasonal flu, but it also varies by location as well as a person's age, underlying health conditions, among other factors. For instance, in Hubei Province, the epicenter of the outbreak, the death rate reached 2.9%, whereas it was just 0.4% in other provinces in China, according to a study published Feb. 18 in the China CDC Weekly.

Scientists aren't certain where the virus originated, though they know that coronaviruses (which also include SARS and MERS) are passed between animals and humans. Research comparing the genetic sequence of SARS-CoV-2 with a viral database suggests it originated in bats. Since no bats were sold at the seafood market in Wuhan at the disease’s epicenter, researchers suggest an intermediate animal, possibly the pangolin (an endangered mammal) is responsible for the transmission to humans. There are currently no treatments for the disease, but labs are working on various types of treatments, including a vaccine. 

Coronavirus disease (COVID-19) outbreak

Coronavirus 

Coronaviruses (CoV) are a large family of viruses that cause illness ranging from the common cold to more severe diseases such as Middle East Respiratory Syndrome (MERS-CoV) and Severe Acute Respiratory Syndrome (SARS-CoV). A novel coronavirus (nCoV) is a new strain that has not been previously identified in humans.

Coronaviruses are zoonotic, meaning they are transmitted between animals and people.  Detailed investigations found that SARS-CoV was transmitted from civet cats to humans and MERS-CoV from dromedary camels to humans. Several known coronaviruses are circulating in animals that have not yet infected humans.

Common signs of infection include respiratory symptoms, fever, cough, shortness of breath and breathing difficulties. In more severe cases, infection can cause pneumonia, severe acute respiratory syndrome, kidney failure and even death.
Standard recommendations to prevent infection spread include regular hand washing, covering mouth and nose when coughing and sneezing, thoroughly cooking meat and eggs. Avoid close contact with anyone showing symptoms of respiratory illness such as coughing and sneezing.

Rahul Gandhi's Response After PM Modi Says He May Quit Social Media

Rahul Gandhi's Response After PM Modi Says He May Quit Social Media

PM Modi, who is a social media celebrity with millions of followers on various platforms, did not elaborate the reason behind his sudden change of mind.
New Delhi: Prime Minister Narendra Modi's surprise announcement of going off social media from Sunday has created quite a flutter on social media. Twitter exploded within minutes of the unexpected tweet that has left everyone speculating.
"This Sunday, thinking of giving up my social media accounts on Facebook, Twitter, Instagram & YouTube. Will keep you all posted," PM Modi tweeted. The tweet was liked over 25,000 times within half an hour.

Soon after, Opposition leaders' reactions started pouring in. Congress leader Rahul Gandhi was one of the first to comment and said, "Give up hatred, not social media accounts."

Tripura Chief Minister and BJP leader Biplab Deb had a response to the former Congress chief's tweet.

"So that's the reason Sonia Gandhi doesn't have any Social Media account?" Mr Deb tweeted tagging Rahul Gandhi's tweet. Rahul Gandhi's mother and current Congress chief Sonia Gandhi is not on social media.

Congress MP Shashi Tharoor also responded to the Prime Minister. He said the abrupt announcement may have led many to speculate whether this is the first step towards banning social media in the country.
Congress spokesperson Randeep Singh Surjewala had a request for the Prime Minister soon after the tweet.
He was soon followed by colleague Alka Lamba who called the move a "big relief".

"Foreign leaders will just miss your congratulatory messages in different languages," she tweeted.

Former Maharashtra Chief Minister Devendra Fadnavis' wife Amruta also replied on Twitter saying that she will "follow the path" of PM Modi, indicating that she too will quit social media.

Former Finance Minister P Chidambaram's son Karti too replied to the Prime Minister saying that the move was "appreciated."

"The toxicity of this medium is a bane," he tweeted.

Gujarat politician Jignesh Mevani also had something to say to the Prime Minister. He asked PM Modi why was he quitting social media after the much publicised "Digital India" programme of the BJP government.

He also urged the Prime Minister to stop following accounts of hate-mongering people who disrespect women instead of quitting social media.

PM Modi, who is a social media celebrity with millions of followers on various platforms, did not elaborate the reason behind his sudden change of mind.

PM Tweet About Leaving Social Media Stirs Up Storm, "No Sir" Is Top Trend

PM Tweet About Leaving Social Media Stirs Up Storm, "No Sir" Is Top Trend

New Delhi: Minutes after Prime Minister Narendra Modi today tweeted he would unplug himself from social media on Sunday, the reactions skyrocketed, with thousands commenting and retweeting in just a matter of minutes.

In half-an-hour, the PM's post was retweeted 9,300 times and got over 27,500 "likes".

"This Sunday, thinking of giving up my social media accounts on Facebook, Twitter, Instagram & YouTube. Will keep you all posted," PM Modi tweeted, leaving people wondering the context of the message.

Thousands of people commented, some asking why he wanted to unplug from some of the world's largest social networking sites, while others posted comments criticising PM Modi over the recent clashes in Delhi that left 46 dead and hundreds injured.

People asked him not to log out, and the words "No Sir" popped up on Twitter's trending list.

"Why sir? We all love to see your posts and updates! Please ese dil mat todiye (don't break our heart)," a Twitter user by the handle Narendra Modi Fan tweeted.

"Why sir? this is the only way to be in touch with you. We know that you are pained with the recent situation but please don't do this. We all know that this phase will go. You are our supporter. We got strength from you. You are our biggest inspiration," Guwahati resident Sanjay D Thakor tweeted.

"We will be missing you Modi ji. Please re-think your decision," Subba Rao, a civil engineer, tweeted.

Yet other users on Twitter took a swipe at PM Modi.

"Sir, please resign as well," tweeted Saniya Sayed, who on her profile says she's a researcher.

The violence in Delhi caused ripples in parliament today as members of the government and the opposition engaged in scuffles, demanded resignation of the Prime Minister and the home minister and caused a series of adjournments.

Web Crawlers and User Agents - Top 10 Most Popular

Web Crawlers and User Agents - Top 10 Most Popular

When it comes to the world wide web there are both bad bots and good bots. The bad bots you definitely want to avoid as these consume your CDN bandwidth, take up server resources, and steal your content. Good bots (also known as web crawlers) on the other hand, should be handled with care as they are a vital part of getting your content to index with search engines such as Google, Bing, and Yahoo. Read more below about some of the top 10 web crawlers and user agents to ensure you are handling them correctly.

Web crawlers#

Web crawlers, also known as web spiders or internet bots, are programs that browse the web in an automated manner for the purpose of indexing content. Crawlers can look at all sorts of data such as content, links on a page, broken links, sitemaps, and HTML code validation.

Search engines like Google, Bing, and Yahoo use crawlers to properly index downloaded pages so that users can find them faster and more efficiently when they are searching. Without web crawlers, there would be nothing to tell them that your website has new and fresh content. Sitemaps also can play a part in that process. So web crawlers, for the most part, are a good thing. However there are also issues sometimes when it comes to scheduling and load as a crawler might be constantly polling your site. And this is where a robots.txt file comes into play. This file can help control the crawl traffic and ensure that it doesn’t overwhelm your server.

Web crawlers identify themselves to a web server by using the User-Agent request header in an HTTP request, and each crawler has their own unique identifier. Most of the time you will need to examine your web server referrer logs to view web crawler traffic.

Robots.txt#
By placing a robots.txt file at the root of your web server you can define rules for web crawlers, such as allow or disallow certain assets from being crawled. Web crawlers must follow the rules defined in this file. You can apply generic rules which apply to all bots or get more granular and specify their specific User-Agent string.

Example 1

This example instructs all Search engine robots to not index any of the website’s content. This is defined by disallowing the root / of your website.

User-agent: *
Disallow: /
Example 2

This example achieves the opposite of the previous one. In this case, the instructions are still applied to all user agents, however there is nothing defined within the Disallow instruction, meaning that everything can be indexed.

User-agent: *
Disallow:
To see more examples make sure to check out our in-depth post on how to use a robots.txt file.

Top 10 web crawlers and bots#
There are hundreds of web crawlers and bots scouring the internet but below is a list of 10 popular web crawlers and bots that we have been collected based on ones that we see on a regular basis within our web server logs.

1. GoogleBot#

Googlebot is obviously one of the most popular web crawlers on the internet today as it is used to index content for Google’s search engine. Patrick Sexton wrote a great article about what a Googlebot is and how it pertains to your website indexing. One great thing about Google’s web crawler is that they give us a lot of tools and control over the process.

User-Agent#
Googlebot
Full User-Agent string#
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Googlebot example in robots.txt#
This example displays a little more granularity pertaining to the instructions defined. Here, the instructions are only relevant to Googlebot. More specifically, it is telling Google not to index a specific page (/no-index/your-page.html).

User-agent: Googlebot
Disallow: /no-index/your-page.html
Besides Google’s web search crawler, they actually have 9 additional web crawlers:

Web crawler User-Agent string
Googlebot News Googlebot-News
Googlebot Images Googlebot-Image/1.0
Googlebot Video Googlebot-Video/1.0
Google Mobile (featured phone) SAMSUNG-SGH-E250/1.0 Profile/MIDP-2.0 Configuration/CLDC-1.1 UP.Browser/6.2.3.3.c.1.101 (GUI) MMP/2.0 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)
Google Smartphone Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Google Mobile Adsense (compatible; Mediapartners-Google/2.1; +http://www.google.com/bot.html)
Google Adsense Mediapartners-Google
Google AdsBot (PPC landing page quality) AdsBot-Google (+http://www.google.com/adsbot.html)
Google app crawler (fetch resources for mobile) AdsBot-Google-Mobile-Apps
You can use the Fetch tool in Google Search Console to test how Google crawls or renders a URL on your site. See whether Googlebot can access a page on your site, how it renders the page, and whether any page resources (such as images or scripts) are blocked to Googlebot.

2. Bingbot#

Bingbot is a web crawler deployed by Microsoft in 2010 to supply information to their Bing search engine. This is the replacement of what used to be the MSN bot.

User-Agent#
Bingbot
Full User-Agent string#
Mozilla/5.0 (compatible; Bingbot/2.0; +http://www.bing.com/bingbot.htm)
Bing also has a very similar tool as Google, called Fetch as Bingbot, within Bing Webmaster Tools. Fetch As Bingbot allows you to request a page be crawled and shown to you as our crawler would see it. You will see the page code as Bingbot would see it, helping you to understand if they are seeing your page as you intended.

3. Slurp Bot#

Yahoo Search results come from the Yahoo web crawler Slurp and Bing’s web crawler, as a lot of Yahoo is now powered by Bing. Sites should allow Yahoo Slurp access in order to appear in Yahoo Mobile Search results.

Additionally, Slurp does the following:

Collects content from partner sites for inclusion within sites like Yahoo News, Yahoo Finance and Yahoo Sports.
Accesses pages from sites across the Web to confirm accuracy and improve Yahoo’s personalized content for our users.
User-Agent#
Slurp
Full User-Agent string#
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)
See Slurp robots.txt documentation.

4. DuckDuckBot#

DuckDuckBot is the Web crawler for DuckDuckGo, a search engine that has become quite popular lately as it is known for privacy and not tracking you. It now handles over 12 million queries per day. DuckDuckGo gets its results from over four hundred sources. These include hundreds of vertical sources delivering niche Instant Answers, DuckDuckBot (their crawler) and crowd-sourced sites (Wikipedia). They also have more traditional links in the search results, which they source from Yahoo!, Yandex and Bing.

User-Agent#
DuckDuckBot
Full User-Agent string#
DuckDuckBot/1.0; (+http://duckduckgo.com/duckduckbot.html)
It respects WWW::RobotRules and originates from these IP addresses:

72.94.249.34
72.94.249.35
72.94.249.36
72.94.249.37
72.94.249.38

5. Baiduspider#

Baiduspider is the official name of the Chinese Baidu search engine’s web crawling spider. It crawls web pages and returns updates to the Baidu index. Baidu is the leading Chinese search engine that takes an 80% share of the overall search engine market of China Mainland.

User-Agent#
Baiduspider
Full User-Agent string#
Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)
Besides Baidu’s web search crawler, they actually have 6 additional web crawlers:

Web crawler User-Agent string
Image Search Baiduspider-image
Video Search Baiduspider-video
News Search Baiduspider-news
Baidu wishlists Baiduspider-favo
Baidu Union Baiduspider-cpro
Business Search Baiduspider-ads
Other search pages Baiduspider
See Baidu robots.txt documentation.

6. Yandex Bot#

YandexBot is the web crawler to one of the largest Russian search engines, Yandex. According to LiveInternet, for the three months ended December 31, 2015, they generated 57.3% of all search traffic in Russia.

User-Agent#
YandexBot
Full User-Agent string#
Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)
There are many different User-Agent strings that the YandexBot can show up as in your server logs. See the full list of Yandex robots and Yandex robots.txt documentation.

7. Sogou Spider#

Sogou Spider is the web crawler for Sogou.com, a leading Chinese search engine that was launched in 2004. As of April 2016 it has a rank of 103 in Alexa’s internet rankings.

Note: The Sogou web spider does not respect the robots.txt internet standard, and is therefore banned from many websites because of excessive crawling.

User-Agent#
Sogou Pic Spider/3.0( http://www.sogou.com/docs/help/webmasters.htm#07)
Sogou head spider/3.0( http://www.sogou.com/docs/help/webmasters.htm#07)
Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)
Sogou Orion spider/3.0( http://www.sogou.com/docs/help/webmasters.htm#07)
Sogou-Test-Spider/4.0 (compatible; MSIE 5.5; Windows 98)

8. Exabot#

Exabot is a web crawler for Exalead, which is a search engine based out of France. It was founded in 2000 and now has more than 16 billion pages currently indexed.

User-Agent#
Mozilla/5.0 (compatible; Konqueror/3.5; Linux) KHTML/3.5.5 (like Gecko) (Exabot-Thumbnails)
Mozilla/5.0 (compatible; Exabot/3.0; +http://www.exabot.com/go/robot)
See Exabot robots.txt documentation.

9. Facebook external hit#

Facebook allows its users to send links to interesting web content to other Facebook users. Part of how this works on the Facebook system involves the temporary display of certain images or details related to the web content, such as the title of the webpage or the embed tag of a video. The Facebook system retrieves this information only after a user provides a link.

One of their main crawling bots is Facebot, which is designed to help improve advertising performance.

User-Agent#
facebot
facebookexternalhit/1.0 (+http://www.facebook.com/externalhit_uatext.php)
facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)
See Facebot robots.txt documentation.

10. Alexa crawler#

Ia_archiver is the web crawler for Amazon’s Alexa internet rankings. As you probably know they collect information to show rankings for both local and international sites.

User-Agent#
ia_archiver
Full User-Agent string#
ia_archiver (+http://www.alexa.com/site/help/webmasters; crawler@alexa.com)
See Ia_archiver robots.txt documentation.

Bad bots#

As we mentioned above most of those are actually good web crawlers. You generally don’t want to block Google or Bing from indexing your site unless you have a good reason. But what about the thousands of bad bots? KeyCDN released a new feature back in February 2016 that you can enable in your dashboard called Block Bad Bots. KeyCDN uses a comprehensive list of known bad bots and blocks them based on their User-Agent string.

When a new Zone is added the Block Bad Bots feature is set to disabled. This setting can be set to enabled instead if you want bad bots to automatically be blocked.

Bot resources#

Perhaps you are seeing some user-agent strings in your logs that have you concerned. Here are a couple of good resources in which you can lookup popular bad bots, crawlers, and scrapers.

BotReports.com

User-Agents.org
Caio Almeida also has a pretty good list on his crawler-user-agents GitHub project.

Summary#

There are hundreds of different web crawlers out there but hopefully you are now familiar with couple of the more popular ones. Again you want to be careful when blocking any of these as they could cause indexing issues. It is always good to check your web server logs to see how often they are actually crawling your site.

What is Search Engine Crawling?

Now that you’ve got a top level understanding about how search engines work, let’s delve deeper into the processes that search engine and web crawlers use to understand the web. Let’s start with the crawling process.

What is Search Engine Crawling?

Crawling is the process used by search engine web crawlers (bots or spiders) to visit and download a page and extract its links in order to discover additional pages.

Pages known to the search engine are crawled periodically to determine whether any changes have been made to the page’s content since the last time it was crawled. If a search engine detects changes to a page after crawling a page, it will update it’s index in response to these detected changes.

How Does Web Crawling Work?

Search engines use their own web crawlers to discover and access web pages.

All commercial search engine crawlers begin crawling a website by downloading its robots.txt file, which contains rules about what pages search engines should or should not crawl on the website. The robots.txt file may also contain information about sitemaps; this contains lists of URLs that the site wants a search engine crawler to crawl.

Search engine crawlers use a number of algorithms and rules to determine how frequently a page should be re-crawled and how many pages on a site should be indexed. For example, a page which changes a regular basis may be crawled more frequently than one that is rarely modified.

How Can Search Engine Crawlers be Identified?

The search engine bots crawling a website can be identified from the user agent string that they pass to the web server when requesting web pages.

Here are a few examples of user agent strings used by search engines:

Googlebot User Agent

Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Bingbot User Agent
Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
Baidu User Agent
Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)
Yandex User Agent
Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)
Anyone can use the same user agent as those used by search engines. However, the IP address that made the request can also be used to confirm that it came from the search engine – a process called reverse DNS lookup.

Crawling images and other non-text files

Search engines will normally attempt to crawl and index every URL that they encounter.

However, if the URL is a non-text file type such as an image, video or audio file, search engines will typically not be able to read the content of the file other than the associated filename and metadata.

Although a search engine may only be able to extract a limited amount of information about non-text file types, they can still be indexed, rank in search results and receive traffic.

You can find a full list of file types that can be indexed by Google available here.

Crawling and Extracting Links From Pages

Crawlers discover new pages by re-crawling existing pages they already know about, then extracting the links to other pages to find new URLs. These new URLs are added to the crawl queue so that they can be downloaded at a later date.

Through this process of following links, search engines are able to discover every publicly-available webpage on the internet which is linked from at least one other page.

Sitemaps

Another way that search engines can discover new pages is by crawling sitemaps.

Sitemaps contain sets of URLs, and can be created by a website to provide search engines with a list of pages to be crawled. These can help search engines find content hidden deep within a website and can provide webmasters with the ability to better control and understand the areas of site indexing and frequency.

Page submissions

Alternatively, individual page submissions can often be made directly to search engines via their respective interfaces. This manual method of page discovery can be used when new content is published on site, or if changes have taken place and you want to minimise the time that it takes for search engines to see the changed content.

Google states that for large URL volumes you should use XML sitemaps, but sometimes the manual submission method is convenient when submitting a handful of pages. It is also important to note that Google limits webmasters to 10 URL submissions per day.

Additionally, Google says that the response time for indexing is the same for sitemaps as individual submissions.

Top 10 Search Engines In The World

Top 10 Search Engines In The World

Which are the 10 best and most popular search engines in the World? Besides Google and Bing, there are other search engines that may not be so well known but still serve millions of search queries per day.
It may be a shocking surprise for many people but Google is not the only search engine available on the Internet today! In fact, there are a number of search engines that want to take Google’s throne but none of them is ready (yet) to even pose a threat.
Recent statistics (updated September 2019), show that Google is the most popular search engine Worldwide with a stunning 81.5% market share.

Nevertheless, there are other search engines worth considering, and the top 10 are presented below.

List of Top 10 Most Popular Search Engines In the World (Updated 2020)

• Google
• Bing
• Yahoo
• Baidu
• Yandex.ru
• DuckDuckGo
• Ask.com
• AOL.com
• WolframAlpha
• Internet Archive

Search Engine Market Share

According to statistics from netmarketshare, statista and statcounter, the top 5 search engines worldwide in terms of market share are Google, Bing, Yahoo, Baidu, and Yandex.

Google is the best search engine with a worldwide market share between 81.5% and 92.96%.

Bing market share is between 2.34% and 5.29%. This places Bing as the best alternative search engine to Google.

Yahoo market share is between 1.64% and 2.04%.

Baidu has a global market share between 0.92% and 9.37% but is the most popular search engine in China.

Yandex, Russian’s most popular search engine has a global market share between 0.47% and 0.83%.

DuckDuckGo market share is between 0.28% and 0.43%.

Google

No need for further introductions. The search engine giant holds the first place in search with a stunning difference of 76% from second in place Bing.

As you can see in the table above, Google is dominating the market in all countries on any device (desktop, mobile, and tablet).

What made Google the most popular and trusted search engine is the quality of its search results. Google is using sophisticated algorithms to present the most accurate results to the users. Google’s founders Larry Page and Sergey Brin came up with the idea that websites referenced by other websites are more important than others and thus deserve a higher ranking in the search results.

Over the years the Google ranking algorithm has been enriched with hundreds of other factors (including the help of machine learning) and still remains the most reliable way to find exactly what you are looking for on the Internet.

Bing

Bing is Microsoft’s attempt to challenge Google in search, but despite their efforts, they still did not manage to convince users that their search engine can be a reliable alternative to Google.

Their search engine market share is constantly below 6%, even though Bing is the default search engine on Windows PCs.

Bing originated from Microsoft’s previous search engines (MSN Search, Windows Live Search, Live Search) and according to Alexa rank is the #30 most visited website on the Internet.

Yahoo

Yahoo is one of the most popular email providers and its web search engine holds the third place in search with an average of 2% market share.

From October 2011 to October 2015, Yahoo search was powered exclusively by Bing. In October 2015 Yahoo agreed with Google to provide search-related services and until October 2018, the results of Yahoo were powered both by Google and Bing. As of October 2019, Yahoo! Search is once again provided exclusively by Bing.

Yahoo is also the default search engine for Firefox browsers in the United States (since 2014).

Yahoo’s web portal is very popular and ranks as the 11 most visited website on the Internet (According to Alexa).

Baidu

Baidu was founded in 2000 and it is the most popular search engine in China. Its market share is increasing steadily and according to Wikipedia, Baidu is serving billions of search queries per month. It is currently ranked at position 4, in the Alexa Rankings.

Although Baidu is accessible worldwide, it is only available in the Chinese language.

Yandex.ru

According to Alexa, Yandex.ru is among the 30 most popular websites on the Internet with a ranking position of 4 in Russian.

Yandex presents itself as a technology company that builds intelligent products and services powered by machine learning. According to Wikipedia, Yandex operates the largest search engine in Russia with about 65% market share in that country.

DuckDuckGo

According to DuckDuckGo traffic stats, they are serving on average 47 million searches per day but still their overall market share is constantly below 0.5%.

Unlike what most people believe, DuckDuckGo does not have a search index of their own (like Google and Bing) but they generate their search results using a variety of sources.

In other words, they don’t have their own data but they depend on other sources (like Yelp, Bing, Yahoo, StackOverflow) to provide answers to users’ questions.

This is a big limitation compared to Google that has a set of algorithms to determine the best results from all the websites available on the Internet.

On the positive side, DuckDuck Go has a clean interface, it does not track users and it is not fully loaded with ads.

Ask.com

Formerly known as Ask Jeeves, Ask.com receives approximately 0.42% of the search share. ASK is based on a question/answer format where most questions are answered by other users or are in the form of polls.

It also has the general search functionality but the results returned lack quality compared to Google or even Bing and Yahoo.

AOL.com

According to netmarketshare the old-time famous AOL is still in the top 10 search engines with a market share that is close to 0.05%.

The AOL network includes many popular web sites like engadget.com, techchrunch.com, and huffingtonpost.com. On June 23, 2015, AOL was acquired by Verizon Communications.

Wolframalpha

WolframAlpha is different than all the other search engines. They market it as a Computational Knowledge Engine which can give you facts and data for a number of topics.

It can do all sorts of calculations, for example, if you enter  “mortgage 2000” as input it will calculate your loan amount, interest paid, etc. based on a number of assumptions.

Internet Archive

archive.org is the internet archive search engine. You can use it to find out how a web site looked since 1996. It is a very useful tool if you want to trace the history of a domain and examine how it has changed over the years.

These are the 10 best and most popular search engines on the Internet today.

The list is by no means complete and for sure many more will be created in the future but as far as the first places are concerned, Google and Bing will hold the lead positions for years to come.

7 Brain Foods for Kids

7 Brain Foods for Kids

As fast as children whiz from classroom to activity to home and back again, their brains are just as rapidly growing and changing.

The foods they eat are important. "These years are critical for brain development, and what they eat affects focus and cognitive skills," says psychiatrist Drew Ramsey, MD, co-author of The Happiness Diet and Fifty Shades of Kale.

These 7 foods can help kids stay sharp and affect how their brains develop well into the future.

7 Brain Foods for Kids

1. Eggs

The protein and nutrients in eggs help kids concentrate, says Los Angeles-based chef Beth Saltz, RD.

How to Serve It:

 Fold scrambled eggs into a whole-grain tortilla for a filling breakfast or late-afternoon snack. "The protein-carb combo tides kids over until the next meal with no sugar-induced energy crash," Saltz says. You can also try serving egg salad sandwiches or a few deviled eggs.

2. Greek Yogurt

Fat is important to brain health, says Laura Lagano, RD. A full-fat Greek yogurt (which has more protein that other yogurts) can help keep brain cells in good form for sending and receiving information.

How to Serve It:

 Pack Greek yogurt in a lunch with some fun mix-ins: cereal with at least 3 grams of fiber, and blueberries for a dose of nutrients called polyphenols. Dark chocolate chips are another option. They have polyphenols, too. These nutrients are thought to keep the mind sharp by hiking blood flow to the brain.

3. Greens

Full of folate and vitamins, spinach and kale are linked to lower odds of getting dementia later in life. Kale is a super food, packed with antioxidants and other things that help new brain cells grow.

How to Serve It:

 For some kids, greens are a hard sell. So rather than serving a salad, you may want to try some different ideas:

• Whip spinach or kale into smoothies for snack time.
• Add spinach to omelets or lasagna.
• Make kale chips. Cut kale from stems/ribs, drizzle with olive oil and a bit of salt, and bake.

4. Fish

Fish is a good source of vitamin D and omega-3s, which protect the brain from declining mental skills and memory loss. Salmon, tuna, and sardines are all rich in omega-3s.

"The more omega-3s we can get to the brain, the better it will function and the better kids will be able to focus," says Bonnie Taub-Dix, RD, author of Read It Before You Eat It.

How to Serve It:

 Grill it and offer your child a sauce for dipping, add fish to tacos, or make tuna sandwiches.

5. Nuts and Seeds

Packed with protein, essential fatty acids, vitamins, and minerals, nuts and seeds may boost mood and keep your nervous system in check.

How to Serve It:

• There’s always peanut butter, or you can also buy or make sunflower seed butter. Sunflower seeds are rich in folate, vitamin E, and selenium, and safe for nut-free zones. If you can’t get your kids to eat the seeds themselves, use the spread on whole-grain crackers or bread.
• Make pesto: Nuts combined with olive oil and dark leafy greens make a healthful and tasty sauce you can serve over whole-grain pasta.

6. Oatmeal

Protein- and fiber-rich oatmeal helps keep heart and brain arteries clear. In one study, kids who ate sweetened oatmeal did better on memory-related school tasks than those who ate a sugary cereal.

How to Serve It: 

Add cinnamon. Compounds in the spice may protect brain cells, research shows.

7. Apples and Plums

Kids often crave sweets, especially when they're feeling sluggish. Apples and plums are lunchbox-friendly and contain quercetin, an antioxidant that may fight decline in mental skills.

How to Serve It: 

The good stuff is often in the skin of fruit, so buy organic, wash well, and put the fruit in a bowl for quick snacks.