Data collection

After identifying the problem and setting the study objectives, the next crucial step is data collection. Most data used in this research were obtained by collaboration with a palm oil mill in Johor, Malaysia. Input parameters, such as POME properties including chemical oxygen demand (COD), total solids (TS) and total volatile solids (TVS), were collected from the mill every month. Additionally, data on temperature, biogas flow rate, H₂S concentrations and biogas composition were collected from their supervisory control and data acquisition (SCADA) system. The data for the machine learning model were collected over a period of two years, which provides detailed operational data for accurate model development and validation.

Data preprocessing

A total of 90 industrial datasets were collected from the palm oil mill in Johor. To generate an effective model for performance analysis, it is essential to preprocess the data to obtain small variations in the output data. Theoretically, the output data usually have a wide range of data. However, if the output range is too large, the ANN model may appear less stable and reliable (27). The preprocessing of the data was carried out by a normalisation technique where the data were scaled to fit within a range of 0-1 using the following equation (28):

/1/

where x is the variable, x_max is the maximum value and x_min is the minimum value.

Overfitting is a common problem that occurs when training neural networks. To solve this problem, the data were divided into three segments: training, testing and validation sets. With this approach, the validation error is constantly monitored and if there is any indication of an increase in it, the training is stopped (28). It was configured so that 70 % of the datasets were used to train the model. Meanwhile, 15 % was set for validation and testing, which confirms the accuracy of the machine learning model. The software used to run this ANN model was MATLAB R2022a (29). The neural network fitting in the software was the application of the programming language used to generate the ANN model.

Process simulation

The simulation of the POME treatment and biogas purification processes was carried out using SuperPro Designer v. 9.0 (18). The input parameters and compositions for the simulation were obtained from the palm oil mill located in Johor, Malaysia. The composition of POME in the feed stream is shown in Table 1. The feed flow rate was obtained from the data collected from the plant. The flow rate of POME from the cooling pond that enters the mixing tank was recorded weekly. The process flow sheet and equipment selection were then simulated using the information provided by the plant. The sizing calculations performed on SuperPro Designer v. 9.0 (18) were completed after mass balance calculations had zero errors. The simulated process flow sheet is shown in Fig. S2.

As shown in Fig. S2, the POME feed enters a mixing tank (P-1) at a flow rate of 14 366 kg/h after leaving a cooling pond. The operating conditions of the mixing tank were set to a temperature of 57.3 °C and a pH=6.89. The mixture is then anaerobically digested in a covered anaerobic lagoon (P-2). The purpose of a covered digester is to contain the produced CH₄ and other gases. The loss of these components could result in a significant loss of profit due to lower purity and emission of greenhouse gases. The biogas produced must then be further purified to remove undesired components such as H₂S. The biogas is purified by removing H₂S in the bioscrubber (P-9).

In this biogas plant, a low-pressure biological scrubber (LPBS) is used for H₂S removal. The H₂S is decomposed by oxidation, producing sulphur and water (13).

An LPBS usually consists of one or more gas-liquid contact columns connected to a recirculation tank consisting of water with nutrients. Each column is partially filled with packing media that are available in a range of shapes (30). The packing media are usually made of non-corrosive materials including glass, metal, ceramic or plastic, on which the microorganisms are immobilised. The shape of the packing media plays an important role in reducing the gas pressure drop as they provide a small contact area per unit volume and increase the void fraction for gas passage (30). Moreover, the packing media allow the microorganisms to form a biofilm (31). The biogas stream is channelled through the media, while the nutrient water is sprayed over them. The nutrient water is sourced from the settling pond after anaerobic digestion and contains only minimal solids (less than 1 %). Contaminants such as H₂S are absorbed from the raw biogas into the liquid, where microorganisms grow on the media and biological oxidation occurs. Normally, bacteria of the genus Thiobacillus are used to remove H₂S from biogas.

Next, the moisture content of this product stream is reduced using a chiller (P-10) (32). In the meantime, the digestate from the anaerobic digester enters a settling pond (P-4) to stabilise wastewater sludge before the bottom sludge is recirculated to the original mixing tank. The treated effluent is then aerobically treated in a facultative pond (P-6) and then in an algal pond (P-7) for further breakdown of biodegradable materials. The pond effluent then enters a discharge pond (P-17), where wastewater is further separated from suspended particles by sedimentation. The sludge is then passed to a dumping pond and sent to a composting plant (33).

Neural network fitting

Machine learning (ML) is a discipline of artificial intelligence (AI) that trains machines or software to identify patterns and make predictions based on historical data with minimal human supervision. Over the years, advances in ML in the industry have shown great potential as a prediction tool in many studies. In the POME industry, it can be observed that the integration of ML is becoming more prevalent in quality estimation in recent years (28).

ANN consists of interconnected nodes or neurons linked by weights. Each node in parallel layers receives data from previous nodes, processes it through a nonlinear function and transmits the result to the subsequent nodes. The modelling process of an ANN involves several steps. The data in this study were recorded weekly and compiled every month from July 2021 to June 2023. A total of 90 datasets were used to develop the machine learning algorithm to obtain the prediction model. With the MATLAB (29) settings for neural network fitting of 70 % for validation and 15 % each for training and testing, the training datasets were used to train the ANN model for future predictions.

Hyperparameters define the structure and learning process of a neural network and play a crucial role in optimising the performance of an ANN model. Unlike model parameters, which are learned during training, hyperparameters such as the learning rate, the number of hidden layers and the number of neurons must be determined before the training process. In this study, random search was used to tune the hyperparameters, as it is more efficient and practical in determining optimal hyperparameter combinations than grid search. This approach helps to prevent overfitting or underfitting and ensures that the model generalises well to unseen data, leading to improved prediction accuracy.

Parameters that affect the performance of the bioscrubber, such as the type of packing medium, bacterial species, empty bed residence time and the recirculation ratio of liquid media (34), play a crucial role. Additionally, other factors such as pressure, biogas flow rate, operating pH and the shape of the packing media can influence the efficiency of the bioscrubber.

The inlet temperature and biogas flow rate were chosen as input data for the model because these parameters are routinely monitored and recorded daily by operators in biogas plants. In particular, the inlet temperature can affect the microbial activity in the bioscrubber, while the biogas flow rate affects the contact time between the biogas and the liquid medium, which affects the efficiency of H₂S removal (35). The outlet H₂S concentration was chosen as the output data and integrated into the output layer of the model. This is because it serves as a direct measure of the performance of the bioscrubber and reflects the effectiveness of H₂S removal in the biogas stream. This concentration is a critical parameter to comply with environmental regulations and is essential for ensuring that the biogas meets quality standards for downstream applications.

Economic evaluation

Techno-economic analysis (TEA) has become a crucial tool for assessing the economic potential of industrial processes, especially in the context of Industrial Revolution (IR) 4.0. As industry has evolved, TEA has increasingly integrated data-driven technologies such as artificial intelligence (AI) and blockchain to optimise processes and economic factors (36). In this study, process simulation is combined with TEA to evaluate the economic feasibility of the biogas plant using the bioscrubber. The analysis includes steps such as process design, equipment sizing, cost estimation and cash flow analysis. Data on plant operation, design, transport and market behaviour were collected from the biogas plant. Market behaviour refers to the trends in pricing and demand for biogas and its by-products, including the feed-in tariff (FiT) policies in Malaysia, which affect the economic viability of the project. After successful simulation and sizing of the equipment, economic calculations were conducted using updated prices from 2022. The economic evaluation function of the SuperPro software, v. 9.0 (18) facilitated the preparation of a comprehensive economic evaluation report.

Statistical analysis

In this study, statistical analysis of mean squared error (MSE) was used as an indicator of the fitness of the model. MSE is determined during the neural network programming using the following equation:

/2/

where o is the observed value for the i-th data point (from experimental or real data) and p is the predicted value generated by the model for the i-th data point.

While the root mean square error (RMSE) is evaluated in the same unit, making it a more interpretable measure, the MSE is more effective in evaluating the distribution of the data and the overall measure of accuracy. This is because outliers have a greater impact as all the differences are squared and all errors are positive. This minimises the effect of negative and positive differences cancelling each other out.

The coefficient of determination (R²) is calculated in ANN modelling using the following equation:

/3/

The fitting of the model improves as R approaches 1.

Simulation model

The simulated results including the composition of biogas from anaerobic digestion and its purified form are shown in Table 2. The simulation results show the final H₂S concentrations in the biogas product to be 43 mg/L and the H₂S removal efficiency by the bioscrubber was calculated to be approx. 95.5 %. This indicates that the operational conditions of the current palm oil mill are satisfactory, as the concentration of H₂S in the biogas remains well below the recommended threshold of 500 mg/L. Overall, the difference of most parameters in Table 2 is less than 10 %, indicating that the simulation model accurately predicts the performance of the bioscrubber system for these parameters.

Results of artificial neural network model

The artificial neural network (ANN) adopts a multi-layer feed-forward neural network that uses the Levenberg-Marquardt training algorithm. The concept of feed-forward is to take measurements and corrective actions before the process is disturbed. The settings and properties of the prediction model are shown in Table 3. The four graphs in Fig. 1 show the regression analysis of the neural network model with desirable outcomes. The values of R² are close to 0.9, indicating that the neural network model is strongly linear with the target values. This model has a great potential to be used in the palm oil industry for prediction purposes that can significantly improve the processes.

Fig. 1

Regression analysis of the ANN model during: a) training, b) validation, c) testing and d) overall phase

The graph in Fig. 2a shows the MSE for training, validation and testing of the generated ANN model as a function of the epoch. An epoch in machine learning can be defined as the total number of iterations of all training data used in one cycle by the algorithm to train the model. The number of epochs is a crucial hyperparameter for the machine learning algorithm. Typical values for the number of epochs range from 10 to 1000 and can be increased further until the model error is sufficiently minimised (37). After running the neural network programme several times, the results with the best R² were used. According to the results of the ANN model, the best validation performance was 0.0067655, which was observed at epoch 11. This means that during training, the neural network reached its optimal performance in the validation dataset at the 11^th iteration of training. After epoch 11, the validation of the data by the model cannot improve further, resulting in overfitting. The ideal ANN model is then developed by focusing on the MSE performance function and the coefficient of determination (R²). The MSE serves as an estimate of the average squared difference between the predicted and actual output values. A lower MSE value means smaller errors in predicting the actual outcomes of the dataset.

Fig. 2

Artificial neural network (ANN) model: a) performance in terms of mean squared error (MSE), b) error histogram with 20 bins

Fig. 2b shows an error histogram with 20 bins from the ANN model, which illustrates the quantitative differences between the target values and the output values after the development of the ANN model. In an error histogram, bins are used to group these differences (errors) into specific ranges, represented as vertical bars. Each bin shows how many predictions fall into a particular error range, which provides a clearer view of the overall accuracy of the model. In this histogram, the model has a notable bin with an error value of -0.00765, which contains around 20 data points where predictions fall within this range. This error value is close to zero, indicating that the model makes predictions with a small margin of error, which suggests a relatively high accuracy. Additionally, the zero-error line on the x-axis represents a perfect match between the predicted and target values, and illustrates the distribution of errors around this ideal point.

Overall, the results of this study highlight the crucial importance of integrating both ANN and SuperPro (18) simulation for effective management of a biogas plant due to their complementary strengths. ANN plays a vital role in accurately estimating specific parameters, such as H₂S removal efficiency, using historical data and complex algorithms based on only two input variables: inlet temperature and biogas flow rate. In terms of performance, the ANN model in this study showed high predictive accuracy, which is confirmed by comparison with the work of Tan et al. (33), who reported an R² value of nearly 1 and an MSE of 0.0002 for the prediction of H₂S concentration. This capability is particularly valuable for optimising process performance and ensuring compliance with gas engine specifications, such as the H₂S concentration in biogas for combustion.

On the other hand, SuperPro (18) simulation extends the predictive power by estimating parameters that are not monitored daily in the plant, such as COD, BOD, TSS, methane content, methane yield, etc. and other operational parameters including hydraulic retention time (HRT), organic loading rate (OLR) and recirculation flow rate. This broader scope enables a comprehensive understanding of overall process efficiency, environmental impact and operational optimisation. By integrating both ANN and SuperPro (18), these tools provide a robust framework for improving biogas plant management, enabling biogas plant engineers to anticipate variations in performance, address potential bottlenecks and take proactive measures to ensure stable and efficient bioscrubber operation. This integrated approach not only improves operational decision-making, but also contributes to the long-term economic viability of the palm oil industry. This aspect is further analysed in the next section.

Techno-economic analysis

The techno-economic analysis (TEA) is a method for evaluating the economic potential of processes in industry (36). The economic analysis for this POME treatment plant was carried out using the economic evaluation function of the SuperPro Designer v. 9.0 software (18). The revenue of the processing plant comes primarily from the sale of the treated biogas, dried sludge and digested sludge to a composting plant. The reliability of this economic evaluation is up to date, as this report uses prices from 2022. Table 4 shows a brief report of the economic evaluation made with the simulation model. According to the sizing calculations performed with the software, the bioscrubber has a volume of 99.39 m³. The unit cost for a plant of this size is estimated at $250 000. The payback period is 7 years, which is consistent with a report by Loh et al. (38). This shows that the bioscrubber is economically feasible and offers a reasonable return on investment.

In contrast, Tan et al. (33) reported a shorter payback period of 5.34 years for a local palm oil mill in Pahang, Malaysia. This shorter payback period was achieved through an optimisation study that emphasised the critical role of optimisation in increasing economic viability. This comparison highlights the potential to improve the effectiveness of the bioscrubber, particularly in increasing the efficiency of H₂S removal. Additionally, optimising the conditions of the anaerobic digester can lead to the production of biogas with lower H₂S concentrations. By focusing on parameters such as feedstock composition and operational conditions, future research can improve both H₂S management and overall biogas quality. As a result, optimisation studies are essential for identifying and implementing strategies that minimise costs and maximise biogas production, which will ultimately contribute to more sustainable practices in the palm oil industry.

Furthermore, excessive H₂S can shorten equipment lifespan and significantly increase maintenance costs (35). The use of an optimised bioscrubber with both ANN and SuperPro (18) will mitigate these effects by lowering H₂S concentrations before the biogas enters critical equipment. This approach would reduce repair costs and extend the lifespan of the infrastructure of a biogas plant, promoting more efficient and cost-effective operations. By improving H₂S management and optimising biogas quality, the integrated strategy ensures that the economic benefits of biogas production are fully realised.

Limitations and future works

Although this study provides valuable insights into the integration of machine learning and SuperPro (18) simulations for biogas purification, it is important to recognise its limitations. A key limitation is the reliance on a dataset of 90 records, collected from a single palm oil mill, which may not capture the full range of variability in POME characteristics across different facilities. Although the dataset was collected under different operating conditions, its relatively small size may limit the generalisability of the model. To mitigate this, future studies could incorporate techniques such as k-fold cross-validation, which would help validate the reliability of the model across different data subsets and reduce overfitting.

The current simulation was limited to certain controllable parameters (e.g. inlet temperature, biogas flow rate), assuming that other operational aspects, such as the biofilm stability or packing media configuration, remain constant. This exclusion of certain biophysical interactions and operational variations may underrepresent how complex, interdependent factors influence bioscrubber efficiency. In future work, including a broader range of parameters or conducting a sensitivity analysis could help identify the primary variables that affect H₂S removal efficiency to better match predictions under real-world conditions.

The performance of both the machine learning model and the SuperPro (18) simulation may also be affected by data quality, plant-specific operational conditions and the limited number of parameters considered in this analysis. Future research should consider extending the dataset to include data from multiple palm oil mills to improve the robustness and the potential to generalise the results. Additionally, conducting a feature importance analysis would help clarify the influence of individual parameters on model predictions, providing a clearer understanding of factors affecting the outputs of an integrated system.

To further improve prediction accuracy, future research could investigate the application of other machine learning models, such as support vector machines or deep learning techniques. Long-term performance evaluation under different seasonal or operational conditions would also be beneficial to evaluate the viability of the integrated approach in real-world settings.