We've just dropped version 21.0 of the Screaming Frog SEO Spider 😱🐸🕷 https://lnkd.in/epHBX4_i This update contains several new features and improvements based upon user feedback and as ever, a little internal steer. 🤖 Direct AI API Integration (OpenAI, Gemini & Ollama) In our version 20.0 release we introduced the ability to connect to LLMs and query against crawl data via custom JavaScript snippets. In this update, you’re now able to directly connect to OpenAI, Gemini and Ollama APIs and set up custom prompts with crawl data. This makes the whole process much easier, as it doesn't require you to edit any JavaScript code, and the SEO Spider will even auto-control the throttling of each model. You can configure up to 100 custom AI prompts and select the category of model, the AI model used, content type and data to be used for the prompt such as body text, HTML, or a custom extraction. In a similar way as custom JS snippets, this can allow you to create alt text at scale, understand the language of a page, detect inappropriate content, extract embeddings and lots more. 🚫Accessibility Auditing You can now perform an accessibility audit in the SEO Spider using the open-source AXE accessibility rule set for automated accessibility validation from Deque. This is what powers the accessibility best practices seen in Lighthouse and PageSpeed Insights, and will help you to improve your websites to make them more inclusive, user-friendly and accessible. The new 'Accessibility' tab details the number of accessibility violations at different levels of compliance based on the Web Content Accessibility Guidelines (WCAG) set by the W3C. You can right-click on any of the violations on the right-hand side, to ‘Show Issue in Browser’ or ‘Show Issue In Rendered HTML’. 📧Email Notifications You can now connect to your email account and send an email on crawl completion to colleagues, clients or yourself to pretend you have lots of friends. You can select to ‘Email on Crawl Complete’ for every crawl to specific email address(es). Alternatively, you can send emails for specific scheduled crawls upon completion via the new ‘Notifications’ tab in the scheduled crawl task as well. The email sent confirms crawl completion and provides some top-level data from the crawl. Fun! 📋 Custom Search Bulk Upload There’s a new ‘Bulk Add’ option in custom search, which allows you to quickly upload lots of custom search filters, instead of inputting them individually. If you’re using this feature to find unlinked keywords for internal linking, for example, you can quickly add up to 100 keywords to find on pages using ‘Page Text No Anchors’. And lots more. Take a look at our release notes for a full list. Enjoy! #seo #technicalseo #seocrawler #ai #chatgpt #accessibility #screamingfrog
Do we still need tokens to connect SF to the AI modules?
S. M. Fahim
3w
Maybe publish a guide with some high level use cases of the custom API. Alt text is not that important, except for the first couple of images on a page. And on WordPress, many plugins auto generate the alt text. I'm sure there could be some high level applications, like content gap analysis, content repurposing for socials, internal link suggestions, etc.
Godwin Josh
3w
The integration of AI into SEO tools like Screaming Frog feels reminiscent of the early days of search engine optimization, when keyword stuffing dominated strategies. Just as we moved away from simplistic tactics, this new wave of AI-powered analysis presents a fascinating opportunity to refine our understanding of content and user intent. Given the potential for bias in large language models, how can developers ensure that AI-driven SEO recommendations promote ethical and inclusive web experiences?
Areej AbuAli
3w
Google: New algorithm update Screaming Frog: Yeah, we'll show you what an actual update means 👏🏽👏🏽👏🏽
Michael King
3w
More 🔥!
Pratik Joshi
3w
No one can match the way Screamingfrog works. While tabiyat other tools are just marketing gimmick. :)
Slide updates needed for Online Expert Days next week 🤪 Nadine McNulty Kai Radanitsch
So I could connect with local LLM via ollama now?
SEO and Content Strategy
3wScreaming Frog Is there a way to only run the prompt (Open AI API) once the crawl and crawl analysis has been finished? For example, I'd like to identify where an internal link is pointing to a page that isn't the "best" destination for the linking text. The 'best' could be defined with embeddings, Ngrams, or other methods. But the prompt is running on every page AS the page gets crawled, so it doesn't know about the other pages it hasn't crawled yet. I'd want to be able to run the prompt once the entire crawl was complete as an internal link analysis. Great update, by the way!