Loss modelling is of critical concern in actuarial science for accurate pricing, reserving, and risk assessment. Claim sizes in a portfolio are heterogeneous due to attributes of policyholders, resulting in behaviors such as multi-modality, skewness, or heavy tails. Similarly, empirical evidence shows that the distribution of claim sizes is different for small, body, and large claim sizes. Hence, traditional parametric methods cannot ac count for all the behavior of losses distribution and give a poor predictive performance. That said, we present a semiparametric severity model in which the whole distribution of claim sizes is learned along with covariates based on a functional regression model for probability densities. We show desirable flexibility and consistency properties of the model and illustrate how it can capture both the distribution and the varying effect of covariates for small and large claims in an interpretative fashion in a real data set in automobile insurance.