Loading custom modules on website pages in sandbox account

[pnylon7]

I cloned a production site account over to a sandbox account. The main reason I did this is so I could test making some changes before doing them in production. Everything seemed to work out up until I had to start recreating pages. I started manually duplicating some web pages but I'm getting stuck at the modules. There are quite a few custom modules and most of them are not visible from the page.

When I navigate to "Content" -> "Website Pages" -> click on the page I want to edit -> click on the "Contents" icon -> then click on "Add new module" under "Main Conent", I don't see the modules I need to load on the page. To make the list shorter, I'll select "Custom modules" from the drop down, but this is where I have to stop. Most of the necessary modules are not in the custom module list.

The directory structure of the current theme is like this:

Theme Name
- css
- js
- modules
- templates

I'm struggling to figure out why those modules are not visible to the pages. I was under the impression that if they were in the theme's module directory, they'd be available just as they were on the live site. I must be missing a small step somewhere along the way. I'm hoping someone else has seen this behavior or knows what's happening.

[pnylon7 · Accepted solution]

I'm not sure if this is the answer that everyone will want to hear, but this is what I heard back from Hubspot support. They said that I will need to add the Content Staging Tool to the subscription and use that instead of the Sandbox. I wasn't able to get any information about the problem I was having with the modules in the Sandbox. I may still look into it further when I have time because I think it could be useful.

[Anton]

Hi @pnylon7, 

a few things you should check:

1. are the modules enabled?

It might have happend, that the modules were set to "not available" during the copying process. (Never seen this before but who knows). You can easily check this by opening a module that's in the theme and check if the "make available in templates and pages" option is enabled. 

If it should be disabled for some reason, enable it and klick on "publish changes"

If you're working with CLI, check the meta.json of the module. The "is_available_for_new_content" option has to be set to true

2. Are the modules available for the correct page type?

Since you're working with a custom theme and custom modules it can be, that the modules aren't available for the page type you're building.

To check this, open the module and check the content types. Here's an example:

If it should say something like "all" or nothing, edit those options

In CLI it looks like this: 

3. Are there disabled modules in the theme.json or templates?

A developer can disable/hide certain modules for a specific theme or even on a template base. This can be done for ease of use for the marketer.

Especially if the theme contains quite an amount of custom modules.

Check your theme.json for something like

"hidden_modules": [
 "@hubspot/button",
 "@hubspot/form"
]

and your templates(first few lines of the template) for 

<--
...
hiddenModules:
- '@hubSpot/follow_me'
- ../modules/form
hiddenSections:
- ../sections/card
-->

and to provide a bit more context about design-tools/design-manager naming:

It's the same function and as long as I remember it was called the design-manager.

Even back in the days when there was only the COS(Content Optimization System). A few years ago HS renamed it to design-tools but it was only for a very short amount of time before they switched back to calling it the design-manager.

Got no official info about that but I assume it's because almost everybody kept calling it design-manager.😀

And maybe - just speculations - they're working on something that will be introduced as the design-tools at a later point. 

best, 

Anton

[pnylon7]

@Anton  Thank you for the detailed information and list to check.

Out of 50+ custom modules in the theme directory only 2 of them are visiblie in the Website Pages (under Main Content) in Hubspot. There are some other custom modules showing up that are in directories outside of the theme directory, but I'm not concerned about those yet.

1. are the modules enabled?
Yes, all the modules appear to be enabled, green with the check switched to the right.
The local meta.json files contain the line "is_available_for_new_content" : true

I didn't check every single module, but saw enough of them to see this result consistantly.

2. Are the modules available for the correct page type?
Yes, this does appear to be set correctly. In Hubspot the modules have the "Content types:" set to - Landing pages, Site pages, Blog listing pages, Blog posts.

Their meta.json files contain:
"content_types" : [ "LANDING_PAGE", "SITE_PAGE", "BLOG_LISTING", "BLOG_POST" ],
"host_template_types" : [ "PAGE", "BLOG_POST", "BLOG_LISTING" ],

I'm assuming that I don't need to turn on everything here.

3. Are there disabled modules in the theme.json or templates?
This I haven't heard of yet, but it looks like I can't find it anywhere.
I don't see "hidden_modules": in any of the template files. I've only been working with one template so far though.

I think I started using Hubspot a little more regularly a short time before they made the latest change to the design manager. At the same time I wasn't sure at first if there was a new design tools they created somewhere else that I hadn't seen yet since manager and tools sound like they could be two different things.

Wel... I don't know where to look next. Should I contact Hubspot support to see if they might know, or should I delete the sandbox abd start over? I'd like not not lose the work I put into it already, but I don't know what else to try. Hopefully I'll get to the bottom of it soon.

Thank you so much for the help!

[Anton]

Hi @pnylon7, 

the HubSpot support could check things differently in the past(don't know how it's now) and might do a thing or two that people outside of HubSpot(like us - the normal user) can't access. 

best, 

Anton

[pnylon7]

Hello @Anton , I sent a support email to Hubspot. If they have a solution to the problem, I will put it here as soon as I find out.

[pnylon7 · Accepted solution]

I'm not sure if this is the answer that everyone will want to hear, but this is what I heard back from Hubspot support. They said that I will need to add the Content Staging Tool to the subscription and use that instead of the Sandbox. I wasn't able to get any information about the problem I was having with the modules in the Sandbox. I may still look into it further when I have time because I think it could be useful.

[Jnix284]

That's a really strange response @pnylon7 - I feel like it's more likely to do with the file paths of the modules in the DnD editor, I forgot, did we confirm the page you were building was using the corresponding theme with the custom modules? or are you trying to make them available within any theme?

[pnylon7]

Yes @Jnix284, I thought so too. I hate to say that opening the support ticket actually made me a little more confused. In the end I wasn't sure if we were still talking about the Content Staging tool or the Sandbox because those two items kept coming back to me interchangeably in the replies when they weren't the current topic in question.

With the Content Staging tool as far as I can tell it looks like you're still connected to the modules in the main/prod account. So if you want to work on the modules you have to create clones of the modules. Then I'm guessing you have to swap them with the original for testing, swap them back when you're done and then copy the finished work to the originl modules to make it live. I haven't used the Content Staging yet, but that seems to be how it would work based on the conversation. Honestly I'm not quite sure how that is different from using the Sandbox or Dev accounts when it comes to modules.

I would think that even if the Sandbox is not considered the preferred way to do "content staging" by Hubspot, there must be a solution that will make the modules load on the page the same way they are doing in the main account.

The Sandbox is supposed to be an imported copy of the main site. After getting the site imported to the sandbox, it was set to use the main site's theme. I had to recreate some pages in Website pages since pages are not imported. The pages in question are all using the same template. The modules that are still missing are inside the modules folder of the current theme. There are a couple custom modules showing up in the Website pages, but I don't see a difference between them and the missing ones. I'm not sure about the file paths of the modules, if they are still the same or if that is something that changes in the sandbox. Does that answer your question?

Thanks!

[Jnix284 · Accepted solution]

@pnylon7 Thanks for the additional details, I agree that Content Staging is not the solution you're looking for, but also the way you want Sandbox to work is also not how it is actually built.

The sandbox is a separate environment, completely disconnected from the main production account.

You can use the CLI to pull your production files to local development, modify them, and then push the changes back.

When you created the files in the Sandbox, you most likely created a separate theme with separate modules - every module has a unique ID that's visible in the backend, this ID is distinct, so if you build a module in Sandbox and bring it across to production, it could end up with a different ID (I haven't done this in a couple of years, so I don't remember exactly, sorry!)

Here's what I would recommend, if you haven't already, join the HubSpot Developer Slack and see if you can get better answers there.

While I've been building websites for over 25 years and using HubSpot for 8 years, I've only started coding/developing with HubSpot in the last 3 years and it's not the focus of my current role, so while I have a lot of experience and context to situations like this, there could be something happening in the background that I haven't run into.

If you want to tag me in the slack at Jennifer Nixon, I can tag a web dev expert I know on the thread to see if he can provide better guidance.

[Jnix284]

@Anton thank you for jumping in, your expertise is very much appreciated!

[pnylon7]

@Jnix284Just wanted to mention that I did a little more research and I still haven't found out what's causing this odd behavior with most of the custom modules. First - I compared one module that's loading to one that isn't and I didn't notice anything missing or out of place. I checked both the files locally and on hubspot. Second - You mentioned I should've uploaded the theme. So even though the theme was already in the Design Manager when  I created the sandbox, I uploaded the theme anyway (cli) to see if anything would change. Unfortunately, I still can't load most of the custom the modules in the Website pages section on Hubspot. At this point I'm not sure what to look at next. 

[Jnix284]

Thanks for the additional details @pnylon7, this isn't something I've run into before, but maybe @Anton has additional insight and can help further. 

[pnylon7]

@Jnix284For this sandbox account, I just used the Hubspot Settings -> Account setup -> Sandboxes. After it was finished copying over the data, I think all the files were in the display manager. I'm pretty sure I didn't have to use the CLI to upload all the files in there. I was assuming that all I had to do after it was created was import the database, upload the necessary images and create/setup the pages.

I also have the other type of dev account where I used the CLI to upload the theme. At first I was thinking to manually set up the dev account for this purpose, but the Sandbox looked like it could be easier or faster to get ready.

[Jnix284]

@pnylon7 based on what you've described so far, if you want to have all of the custom modules and dev files, the sandbox export/import won't be enough and you'd need to use the CLI to have full access - I've never seen a sandbox creation fully copy design tools, but it's been a while and perhaps that's changed (doesn't sound like it since your custom modules are missing)

[pnylon7]

@Jnix284I'm sorry. I should have mentioned that all the files were visible in the Design Manager immediately after the sandbox was created. The process must be copying all of them over now. I can view and edit all the theme files, including the modules, from within the Design Manager. That's why I didn't use the CLI to upload anything. The files are there, but the they are not visible from the Website Pages. When I create a page and try to "Add new module" the modules that are needed for the page are not in the that list. That popup window's list of modules seem to be random. I only see a few of the modules from the theme directory and then some other modules that are in different directories. Actually, that must be the key to figuring this out. Why are only some modules randomly showing up in the Add new modules list for the Website pages? I guess I just need to find out what is different about the modules that are loading in there. I'm not sure what that could be yet.

BTW, I really appreciate you taking the time to help me sort this out. Thank you.

[Jnix284]

@pnylon7 are the custom modules in a child theme? or another folder/directory in design tools? A custom module doesn't have to be in the main theme folder to be used in a theme.

[pnylon7]

Thanks for the quick reply. You must mean the Design Manager because that's where I'm seeing the files. Design Tools must be from an older menu?

The theme the site is using now was already there before I arrived. It appears to be a custom theme. I don't think I've created a child theme yet so I'm not sure what they look like compared to a parent. The Design Manager is cluttered with a lot of files so I'm not yet sure what's being used. I know which theme is selected for the site and it looks like the necessary modules are in that theme folder, although there are modules in other folders that could be in use. I'd have to look into that. It also looks like there are older versions of the theme that are no longer being used.

To answer your question, it looks to me like all or most of the custom modules that are in use on the live site's pages are in the same main theme folder that is currently selected for the site. I see them loaded on the left side of the Website pages in the live site, but I still can't select them in the sandbox when creating a page. It looks like none of them are loading for the pages.

[Jnix284]

@pnylon7 yes, sorry, design tools is what it was previously called, design manager is the right place to check.

How did you copy the theme from production to sandbox? with the CLI?