Mediterranean diet (MD) is among the most commonly investigated diets and recognized as one of the healthiest dietary patterns. Due to its complexity, geographical and cultural variations, it also represents a challenge for quantification. The aim of this cross-sectional study was to assess reliability and validity of the Croatian version of the 14-item Mediterranean Diet Serving Score (MDSS), using the Mediterranean Diet Adherence Screener (MEDAS) as the referent test. We included the exploratory sample of 360 medical students, and a confirmatory sample of 299 health studies students from the University of Split, Croatia. Test-retest reliability and validity of the MDSS were tested using intra-class correlation coefficients (ICC), while Cohen's kappa statistic was used to test correct classification of subjects into MD adherent/non-adherent category. A very good reliability was shown for the overall MDSS score (ICC = 0.881 [95% CI 0.843-0.909]), and a moderate reliability for the binary adherence (κ = 0.584). Concurrent validity of the MDSS was also better when expressed as a total score (ICC = 0.544 [0.439-0.629]) as opposed to the adherence (κ = 0.223), with similar result in the confirmatory sample (ICC = 0.510 [0.384-0.610]; κ = 0.216). Disappointingly, only 13.6% of medical students were adherent to the MD according to MDSS, and 19.7% according to the MEDAS questionnaire. Nevertheless, MDSS score was positively correlated with age (ρ = 0.179: P = 0.003), self-assessed health perception (ρ = 0.123; P = 0.047), and mental well-being (ρ = 0.139: P = 0.022). MDSS questionnaire is a short, reliable and reasonably valid instrument, and thus useful for assessing the MD adherence, with better results when used as a numeric score, even in the population with low MD adherence.