Borderline impossible to run Solr in a development environment
It seems that recently the Solr containers in Upsun fixed have changed, specifically they now seem to consume a lot more RAM than they did before.
They also seem to kill the process when it runs out of allocated RAM.
This is making it almost impossible to work with Solr in development environments in my projects. The services are always scaled to ‘S’ and are usually using at least 200% of their memory allocation, if not more.
This is is fairly standard Drupal + Solr setup, with the default config coming from the Drupal SearchAPI Solr module, so I'm not doing anything particularly exciting I think.
This is latest Solr 9, but I also observed it with an older Solr 8 version too.
What I'm seeing is that if I send some items for indexing to Solr then it very quickly runs out of RAM and is then killed, and then Solr restarts, but spends several minutes doing so. Very frustrating!
Has something changed in the setup of these containers recently? Because the workflow of using the development environments to do, you know, development and test things out w.r.t. Solr was working fine some time ago.
Comments
In short, yes, there have been some changes.
When we introduced Flex, shortly after, we also made some changes to what is allowed on Fixed.
Previously, on Fixed, we allowed a significant burst of resources, where in some cases, projects and containers were able to consume over 200% of the resources they were paying for. Initial intetion there was to allow projects to handle short bursts of traffic or resource-heavy tasks. With the introduction of Flex, and the idea of flexible resources and, pay for resources you commit, where a project can commit various combinations of resources, making it resource-specific for the required demand, we decided to try to be fair in terms of resource allocation between Flex and Fixed, and not allow extensive overcommitment (over 100%) of resources on Fixed.
Flex has resolved this situation of development environments being limited to a specific size. We do allow some burst for Enterprise and Elite Tier projects in some specific situations on Fixed.
Generally, all new Fixed projects do not have bursts anymore, possibly having some issues in development environments for some use cases. For projects that require more resources in dev environments, we recommend moving to Flex projects.
Please sign in to leave a comment.