Developers at Yoast and Google Collaborate on Proposal to Add XML Sitemaps to WordPress Core

Developers at Google and Yoast are collaborating with other contributors on a proposal to add XML sitemaps to WordPress core. This capability has traditionally been handled by plugins, which provide a variety of implementations. The goal of this new feature project is to ship basic XML sitemaps in core while providing an XML Sitemaps API for plugin developers to extend. It would also update WordPress’ robots.txt file to reference the sitemap index.

Thierry Muller, a Developer Relations Program Manager at Google, published the details of the collaborators’ plans on WordPress.org. The sitemaps included in core would be enabled by default and would index the following content:

  • Homepage
  • Posts page
  • Core Post Types (Pages and Posts)
  • Custom Post Types
  • Core Taxonomies (Tags and Categories)
  • Custom Taxonomies
  • Users (Authors)

Although WordPress contributors have had multiple discussions regarding XML sitemaps on trac, Muller said there had not yet been a group to take ownership and get things moving forward. His proposal notes that 4 out of the top 15 plugins in the WordPress plugin directory ship with their own implementation of XML. Despite the demonstrated high demand for XML sitemaps, the right team for getting this added to core had not been assembled until now.

“XML Sitemaps is a standard across all search engines these days and any website with content which is intended to be indexed should have one,” Muller said. “Having XML Sitemaps in core would speed up content discoverability and indexing. We started discussing this topic last year with Yoast and agreed that it would be great to finally make it happen.”

So far the initiative has received a favorable response from the community and has already gained the support of WordPress project lead Matt Mullenweg.

“This makes a lot of sense, looking forward to seeing the v1 of this in core and for it to evolve in future releases and cement WordPress’ well-deserved reputation of being the best CMS for SEO,” Mullenweg commented on the proposal.

Performance is one of the primary concerns that the WordPress development community has expressed regarding adding XML sitemaps to core.

“As simple and straightforward as XML sitemaps are, they present some relatively significant performance challenges at scale,” Matthew Boynes, partner at Alley Interactive, said. “As one for-instance, how many urls are going to be in each paginated (sub-) sitemap? A sitemap index file is limited to 50,000 sitemaps, so even though each sitemap is limited to a maximum of 50,000 urls, generating 50,000 urls in one page request would be extremely difficult and non-performant to do on-the-fly.”

Muller and other contributors on the project have a strategy for addressing scalability concerns and are planning to keep these considerations on the forefront while building core’s implementation.

“As for everything else, engineering new features should always be done with security and performance in mind,” Muller said. “It starts at the high level architecture down to the granular details. Each Sitemap entry will only contain , and which should not overload the server request with a paginated approach. The number of entries per Sitemap is still to be defined, performance will definitely be a deciding factor.”

The XML Sitemaps project has a clearly defined scope. The initial implementation will not include features like image, video, or news sitemaps. It also will not include a UI for controls to exclude individual posts, pages, or custom post types from the sitemap. This is somewhat controversial but plugins can always extend core to offer these features until a more sophisticated version of sitemaps lands in core.

Plugin authors who offer XML sitemaps will need to re-architect their plugins with the new API in order to avoid conflicts and indexing errors.

“I can confidently say that we will make it as smooth as possible and work with plugin authors to avoid conflicts,” Muller said. “If anything else, having XML Sitemaps as well as a Core API to extend it will make their lives easier – plugin authors will have a standard way to extend it.”

This effort to get sitemaps added to WordPress core is not limited to those working at Google and Yoast. Muller encouraged authors of other XML sitemaps plugins to get involved or at least follow the project’s development to ensure their plugins are compatible once it is merged into core.

WordPress 5.2.2 Squashes 13 Bugs

WordPress 5.2.2 dubbed a short-cycle maintenance release is available for download. This release addresses 13 bugs and improves the Site Heath features introduced in 5.2.

One of the changes in 5.2.2 affects the theme update link in the Customizer of multisite installs. The update link for themes on multisite installs in the Customizer has been removed. This is because updates can not be performed from within the Customizer in this situation.

Other changes include, adding a exit recovery mode button in the responsive view, making the Site Health page access filterable, and updates to the page editor packages.

JB Audras, Justin Ahinon andMary Baum co-led the release with guidance from Executive Director, Josepha Haden. Thirty people contributed to WordPress 5.2.2.

In Case You Missed It – Issue 26

In Case You Missed It Featured Image
photo credit: Night Moves(license)

There’s a lot of great WordPress content published in the community but not all of it is featured on the Tavern. This post is an assortment of items related to WordPress that caught my eye but didn’t make it into a full post.

Birgit Olzem Could Use the Community’s Help

Birgit Olzem has encountered some financial hard times due to multiple illnesses and paying for acupuncture treatments and osteopathic therapy which are not covered by her insurance. Olzem fell ill earlier this year which prevented her from generating income as a self-employed person.

Olzem has translated WordPress, compiled release packages for de_DE, organized Meetups, WordCamps, answered support questions and has been part of different make/WordPress teams, some of them as a team-rep. She’s also contributed to WordCamps as a Speaker, Volunteer and Contributor day team-lead.

To learn more about her story and to donate, please read her request to the community.

Liam Dempsey’s Take on Gutenberg

Liam Dempsey describes what he likes and doesn’t like about the new WordPress editor.

Defending the Right to Publish Open Source Software in the UK

The EFF and Open Rights Group have submitted comments to the UK government defending the right to publish open source software.

Moreover, source code is a form of written creative expression, and open source code is a form of public discourse. Regulating its publication under anti-money-laundering provisions fails to honor the free expression rights of software creators in the United Kingdom, and their collaborators and users in the rest of the world.

Why Is It Important to Give Back to Open Source?

JC Mae Palmes on Twitter asked, why is it important to give back to the WordPress community? Here are a few responses. To see all of the responses, check out this Twitter thread

WordCamp US Speaker Selection Process

If you’re wondering how speakers are being selected for WordCamp US this year, check out this post by the Programming Team. The team is using a new process that includes, limiting the number of submissions per speaker to two instead of unlimited, reviewing submissions based on the organizer’s sphere of experience, and using blind reviews. Speakers who are chosen are scheduled to be notified by the end of this month.

An Easy Way to Make an Impact in The WordPress Community

David Bisset shared the following idea on Twitter and while a few companies have started doing this, I think it will catch on with individuals more than businesses.

That’s it for issue twenty-six. If you recently discovered a cool resource or post related to WordPress, please share it with us in the comments.

Justin Tadlock Proposes Idea to Solve Common Theme Issues

The Theme Review Team has been discussing ideas in Slack on how to solve the problem of themes in the review queue suffering from common theme issues. Just Tadlock has proposed a idea he calls Theme Feature Repositories.

The idea is to create standardized packages on the Theme Review Team GitHub repo that authors could use in their themes. If enough people bought into the idea and worked together, it would lessen the pain points between reviewers and theme authors. It would also decrease the amount of code written by hundreds of different authors to solve a common problem.

Tadlock used Admin notices and Links to ‘Pro’ versions as two examples that could benefit from this approach. Packages would handle specific use cases and be installed using Composer. For those who don’t use composer, an autoloader would be provided as well as a .zip file that could be dropped into a theme.

Tadlock is asking the theme community what packages do they need or what common problems could be solved together.

“This can literally be any common feature in WordPress themes, not just admin or customizer-related things,” Tadlock said. “Nothing is ‘out of bounds’. Every idea is on the table right now.

“This is an ambitious project. It’d require cooperation between authors and reviewers for the betterment of the theme directory as a whole. It’ll only work if we have buy-in from everyone.”

Tadlock also mentioned that due to his schedule, he will be unable to lead or co-lead the project and is seeking people interested in taking on these roles. Those interested should have knowledge of Git, Composer, and Object-oriented programming.

If you’re interested in this project or want to provide feedback, you can leave a comment on the proposal.