Backbone.js and Collections: Plugin compatibility

Ensuring plugin compatibility within the Single Page Application in Collections was a major challenge. In this post, I will discuss some of the measures we took to increase the probability of plugin compatibility in Collections.

JSON and filters

When a new page is loaded in the Collections Single Page Application (SPA), an AJAX request is made to the server, which returns JSON data that is sent to a JavaScript (JS) template and rendered on the page. Since WordPress does not provide a JSON API, a JSON endpoint was created for use with Collections. We specifically decided to build a limited JSON endpoint into Collections because we wanted to increase the probability of plugin compatibility in the theme.

In WordPress, data in the database is not the final form of that data. Due to filters, the data can be altered via PHP code, which allows the final state of the data to be different than its original state in the database. This is a popular way for plugins to change post content as it does not permanently change the content. Furthermore, WordPress shortcodes can be saved within a post’s content, which are transformed to content when printed. We felt strongly that Collections needed to preserve these conventions while employing the SPA particularly because so many plugins use these mechanisms.

Collections relies on this data being prepared on the server before being sent back to the client side application. In other words, the data is retrieved from the database, filtered, then returned to the application. To illustrate how this is handled in the JSON template, see the extracted code below:

if ( have_posts() ) {
    while( have_posts() ) {
        // Make sure that the content is properly processed before printing
        $content = apply_filters( 'the_content', get_the_content(), get_the_ID() );
        $content = str_replace( ']]>', ']]>', $content );

        // Same with the excerpt
        $excerpt = apply_filters( 'the_excerpt', get_the_excerpt() );

        $data = array(
            'content'     => $content,
            'excerpt'     => $excerpt,
            'bodyClasses' => join( ' ', get_body_class() ),
            'postClasses' => join( ' ', get_post_class() ),
        );
    }
}

...

echo json_encode( $data );

The post content is processed identically to how the_content() handles the post’s content. Since Collections is returning data the same way that the data would normally be returned, the probability of plugin compatibility is greatly increased.

Additionally, because changing views would change other facets of a view’s HTML, other data is returned. In the example above, the page’s body classes and post classes are returned. These are used to replace the current body and post classes in the page. Since plugins can also alter these attributes, it is important they are considered when accessing the data.

It is important to remember that getting data from the database is only half of the battle. The JSON API/endpoint implemented also needs to consider other plugins and code filtering the data before it is used. Otherwise, plugin compatibility will be compromised.

I would have gotten away with it if it wasn’t for you meddling enqueues

Filtering content is a huge step toward plugin compatibility, but it leaves out an important detail: additional scripts and stylesheets. Scripts and stylesheets, collectively know in WordPress as enqueues, change on a page by page basis. For instance, an audio post in Collections relies on the MediaElement.js library, which includes extra JS and CSS. When an audio post loads via the SPA, those scripts need to load for the audio to work correctly. There are countless reasons why additional scripts are needed and Collections tries to figure this out in order to support core and plugin functionality.

To solve this problem, we looked at prior art. Infinite Scroll, as implemented in Jetpack and on WordPress.com, has successfully solved this problem for sometime. Similarly, Infinite Scroll needs to handle additional scripts when new posts are scrolled into view. Given that Infinite Scroll is quite successfully running on millions of websites, we thought this might be a good place to start when working through this problem.

Breaking this down into small pieces, the logic behind the solution to this problem is as follows:

1. When the initial page loads, record, in an object accessible via JS, which enqueues have been loaded.
2. Upon making a request for a new page, set an object indicating which scripts need to be loaded for the page request.
3. Determine which enqueues are not yet loaded and load them.

Let’s take a close look at each of these steps.

Record initial enqueues

This part of the solution was the easiest. All that was needed was introspecting the $wp_scripts and $wp_styles variables to determine what enqueues were loaded for the page view. Once the application has that information, it needs to be set to JS objects. We accomplished this with the following code:

function collections_denote_loaded_scripts_and_styles() {
    global $wp_scripts, $wp_styles;

    $scripts = ( is_a( $wp_scripts, 'WP_Scripts' ) ) ? $wp_scripts->done : array();
    $styles  = ( is_a( $wp_styles, 'WP_Styles' ) ) ? $wp_styles->done : array();
?>
    <script type="text/javascript">
        collectionsSPAData.scripts = ;
        collectionsSPAData.styles = ;
    </script>
<?php
}
endif;

add_action( 'wp_footer', 'collections_denote_loaded_scripts_and_styles', 20 );

It is important to note that the collectionsSPAData object was previously defined using the wp_localize_script function. Unfortunately, we could not use that function for generating this data because the data generation needed to happen well after the function would run. In fact, we intentionally run this at priority 20 on the wp_footer action to increase the likelihood that all enqueues are handled by the time we run this code.

With this done, the client side application can use collectionsSPAData.scripts and collectionsSPAData.styles to determine which scripts and styles are loaded, respectively.

Determine new enqueue requirements

When a request for the JSON data for a new view is made, the application needs to assess the needed dependencies for the page. Doing this is tricky, feels hacky, and certainly is not what any developer wants to be doing; however, the method used in Jetpack and altered in Collections has proven to be quite effective.

WordPress only knows what scripts have been enqueued once the wp_enqueue_scripts() function and its residual functionality is completed. The wp_enqueue_scripts() function is hooked to wp_head and will cause functions to also execute on the wp_footer action. As such, the most reliable way to determine the required enqueues is to emulate the loading of wp_head() and wp_footer() and inspect the impact that this had on the enqueues.

To achieve this goal, the template used for the JSON data spoofs a full page load without allowing the contents of the wp_head() or wp_footer() functions get written to the page:

ob_start();
wp_head();
ob_end_clean();

if ( have_posts() ) {
    while( have_posts() ) {
        the_post();
        
        // Get the data
    }
}

ob_start();
wp_footer();
ob_end_clean();

Because the functions have been executed, the data in $wp_scripts and $wp_styles has been set and it can be inspected to determine the required enqueues for the page view:

// Get the scripts/styles that were rendered
$scripts = ( is_a( $wp_scripts, 'WP_Scripts' ) ) ? $wp_scripts->done : array();
$styles  = ( is_a( $wp_styles, 'WP_Styles' ) ) ? $wp_styles->done : array();

$enqueues = array(
    'scripts' => $scripts,
    'styles'  => $styles,
);

// Get all of the necessary script data for each script
$scripts = array();
foreach ( $enqueues['scripts'] as $handle ) {
    $scripts[ $handle ] = collections_get_script_data( $handle );
}

// Get necessary styles data for each stylesheet
$styles = array();
foreach ( $enqueues['styles'] as $handle ) {
    $styles[ $handle ] = collections_get_style_data( $handle );
}

This snippet uses the same strategy used earlier to determine which enqueues were present when the page loads. The done property contains the handle for the enqueued script or style. Then, this code gets all of the additional data associated with the enqueue (e.g., load in header or footer, version, dependencies, etc.).

These values are appended to the JSON and passed to the client side application, which can then handle loading the enqueues.

Loading the enqueues

Once the AJAX response is completed, the application is aware of the required scripts and styles that need to be loaded on the page; however, in order to be efficient, it should only load the enqueues that are not already loaded. The application checks to see if a script has been previously loaded and only loads it if it has not been loaded. As example, the following shows the function that loads the needed scripts:

_renderScripts = function ( scripts ) {
    _.each( scripts, function( list, iterator ) {
        // Add script handle to list of those already parsed
        collectionsSPAData.scripts.push( list.handle );

        // Determine where to load the script
        var where = ( list.footer ) ? 'body' : 'head';

        // Output extra data, if present
        if ( list.extra_data ) {
            var data        = document.createElement('script'),
                dataContent = document.createTextNode( "//<![CDATA[ \n" + list.extra_data + "\n//]]>" );

            data.type = 'text/javascript';
            data.appendChild( dataContent );

            document.getElementsByTagName( where )[0].appendChild( data );
        }

        // Load the script and trigger a callback function when the script is fully loaded and executed
        _loadAndExecute( list.src, where, list.handle, function() {
            $( window.document ).trigger( 'post-load' );
        } );
    } );
};

_loadAndExecute = function( src, where, handle, callback ) {
    // Create the script element
    var script = document.createElement( 'script' ),
        loaded;

    // Append the necessary source
    script.setAttribute( 'src', src );

    // If there is a callback, run it when the script is loaded
    if ( callback ) {
        script.onreadystatechange = script.onload = function () {
            if ( ! loaded ) {
                callback();
            }
            loaded = true;
        };
    }

    // Append the script
    document.getElementsByTagName( where )[0].appendChild( script );
};

While that is a lot of code, the basic approach is that each script is inspected. If it has not been loaded, it is prepared to be loaded. The script is then loaded in either the head or footer of the document. Additionally, a callback is triggered once the script is loaded in order to trigger functionality after the script is loaded. Styles are loaded in a similar fashion with the only difference being how they are appended to the DOM.

With these enqueues loaded and executed, the enqueues are present on the page as the developer intended. This helps ensure that the original experience is preserved even though the content is loaded dynamically, rather than on a page refresh.

Plugin compatibility is hard

We knew from the outset that making Collections compatible with plugins would be a challenge. We absolutely could not dream of shipping a theme that would break a vast amount of plugins in the community. I am sure that there are many plugins that Collections would not be compatible with; however, we have gone to great lengths to try to make the theme compatible with plugins.

Plugin compatibility represents one of the challenges of building a client side application in a server side world. In the next and final post in this series, I will discuss the idiosyncrasies of trying to adapt a server side application to a client side application.

Enjoy this post? Read more like it in From the workshop.