Many models of different complexities are developed for fisheries stock assessment, and yet few have been rigorously evaluated for their performance in capturing fisheries population dynamics. This causes confusion about when a model should be used or not in assessing fisheries resources. This is especially true for models with complex structures. The present study evaluated the performance of a seasonal, sex-specific and size-structured stock assessment model with respect to the temporal pattern of recruitment, observation errors associated with input data, process errors and violation of model assumptions for the American lobster Homarus americanus. Using an individual-based lobster simulator, a series of lobster fisheries with different characteristics were simulated and the model was applied to the simulated data to estimate key fisheries parameters. Estimated values were then compared with the true values in the simulated fisheries to evaluate the model’s ability to capture the temporal trend in stock abundance, biomass and recruitment, and to identify factors that might result in model failure. Results show that this newly developed lobster stock assessment model performs well in capturing the dynamics of the lobster population under a wide range of conditions. Temporal trends in natural mortality and biased estimates of growth parameters posed the most serious problems. The present study shows the importance of model evaluation.