While developing the little script that allowed me to use Pushover to receive YouTube statistics I came across Yahoo Query Language.
Yahoo provides some very interesting services. I remember using Yahoo! Pipes a very long time ago and I was amazed by it’s simplicity. It was perfect for building data mashups that aggregate web feeds, web pages, and other services, creating Web-based apps from various sources, and publishing those apps. I learned a whole lot about scraping the web, but sadly the service was shut down on 30 September 2015.
While working on my latest Pushover script I wanted to pull some statistics from YouTube. One issue is that the YouTube API requires a developer API key which I didn’t want to depend on. I also didn’t want to rely on other external libraries for screen scraping my channel page. After looking around the web for alternative approaches I then found this fun lille project displaying YouTube statistics in real-time… but how?
https://query.yahooapis.com/v1/public/yql?q=SELECT * FROM html where url="https://www.youtube.com/channel/UCW_wx3mNy3EiaXljR_iVVUQ/about" AND xpath='//*[@class="about-stats"]//b'&format=xml
The code is simple and self-explanatory. If you navigate to my YouTube channel’s about page (don’t forget to subscribe) you’ll see both the subscriber and view count. Then it uses XPath to extract the bold text (b) from any HTML element that has the class “about-stats” assigned. You can also tell Yahoo to output the result in xml or json.