Publications

You can also find my articles on my Google Scholar profile.

Conference Papers

A Human Perspective to AI-based Candidate Screening.

Published in 58th Hawaii International Conference on System Sciences (HICSS), Hawaii, US., 2025

Skill extraction is at the core of algorithmic hiring. It is based on identifying terms commonly found in both targets (i.e., resumes and job offers), aiming at identifying a “match” or correspondence between both. This paper focuses on skill extraction from resumes, as opposed to job offers, and considers this task both from the Human Resource Management (HRM) and AI points of view. We discuss challenges identified by both fields and explain how collaboration is instrumental for a successful digital transformation of HRM. We argue that annotation efforts are an ideal example of where collaboration between both fields is needed and present an annotation effort on 46 resumes with 41 trained annotators, resulting in a total of 116 annotations. We analyze the skills extracted by multiple different systems and compare those to the skills selected by the annotators, and find that the skills extracted differ a lot in terms of length and semantic content. The skills extracted with conversational Large Language Models (LLMs) tend to be very long and detailed, other systems are very concise, whereas humans are in the middle. In terms of semantic similarity, conversational LLMs are closer to human outputs than other systems. Our analysis proposes a different perspective to understand the well-studied, but still unsolved skill extraction task. Finally, we provide recommendations for the skill extraction task that aligns with both HR and computational perspectives

Recommended citation: Vásquez-Rodríguez, L., Audrin, B., Michel, S., Galli, S., Rogenhofer, J., Cusa, J. N., & van der Plas, L. (2025). A Human Perspective to AI-based Candidate Screening. 58th Hawaii International Conference on System Sciences (HICSS). Hawaii, US.
Download Paper

Simple is not Enough: Document-level Text Simplification using Readability and Coherence

Published in arXiv preprint., 2024

We present a coherence-aware evaluation of document-level Text Simplification (TS), an approach that has not been considered in TS so far. We improve current TS sentence-based models to support a multi-sentence setting and the implementation of a state-of-the-art neural coherence model for simplification quality assessment. We enhanced English sentence simplification neural models for document-level simplification using 136,113 paragraph-level samples from both the general and medical domains to generate multiple sentences. Additionally, we use document-level simplification, readability and coherence metrics for evaluation. Our contributions include the introduction of coherence assessment into simplification evaluation with the automatic evaluation of 34,052 simplifications, a fine-tuned state-of-the-art model for document-level simplification, a coherence-based analysis of our results and a human evaluation of 300 samples that demonstrates the challenges encountered when moving towards document-level simplification.

Recommended citation: Vásquez-Rodríguez, L., Nguyen, N. T., Przybyła, P., Shardlow, M., & Ananiadou, S. (2024). Simple is not Enough: Document-level Text Simplification using Readability and Coherence. arXiv preprint arXiv:2412.18655.
Download Paper

Hardware-effective Approaches for Skill Extraction in Job Offers and Resumes.

Published in RecSys in HR’24: The 4th Workshop on Recommender Systems for Human Resources, in conjunction with the 18th ACM. Conference on Recommender Systems, Bari, Italy., 2024

Recent work on the automatic extraction of skills has mainly focused on job offers and not resumes while using state-of-the-art resource-intensive methods and considerable amounts of annotated data. However, in real-life industrial contexts, the computational resources and the annotated data available can be limited, especially for resumes. In this paper, we present our experiments that use hardware-effective methods and circumvent the need for large amounts of annotated data. We experiment with various methods that vary in hardware requirements and complexity. We evaluate these systems both on public and commercial data, using gold-standard for evaluation. We find that standalone rule-based and semantic model performance on the skill extraction task is limited and variable between job offers and resumes. However, neural models can perform competitively and be more stable, even when using small datasets, with an improvement of ∼30%. We present our experiments using minimal hardware, mostly CPU-based with less than 8 GB of RAM for rule-based and semantic methods and using GPUs for neural models with a maximum memory usage for both CPU and GPU of 24 GB, with less than 25 minutes of training time.

Recommended citation: Vásquez-Rodríguez, L., Audrin, B., Michel, S., Galli, S., Rogenhofer, J., Cusa, J. N., & van der Plas, L. (2024). Hardware-effective Approaches for Skill Extraction in Job Offers and Resumes. RecSys in HR’24: The 4th Workshop on Recommender Systems for Human Resources, in conjunction with the 18th ACM Conference on Recommender Systems, Bari, Italy.
Download Paper | Download Slides

BLESS: Benchmarking Large Language Models on Sentence Simplification.

Published in In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. Singapore. Association for Computational Linguistics., 2023

We present BLESS, a comprehensive performance benchmark of the most recent state-of-the-art Large Language Models (LLMs) on the task of text simplification (TS). We examine how well off-the-shelf LLMs can solve this challenging task, assessing a total of 44 models, differing in size, architecture, pre-training methods, and accessibility, on three test sets from different domains (Wikipedia, news, and medical) under a few-shot setting. Our analysis considers a suite of automatic metrics, as well as a large-scale quantitative investigation into the types of common edit operations performed by the different models. Furthermore, we perform a manual qualitative analysis on a subset of model outputs to better gauge the quality of the generated simplifications. Our evaluation indicates that the best LLMs, despite not being trained on TS perform comparably with state-of-the-art TS baselines. Additionally, we find that certain LLMs demonstrate a greater range and diversity of edit operations. Our performance benchmark will be available as a resource for the development of future TS methods and evaluation metrics.

Recommended citation: Kew, T., Chi, A., Vásquez-Rodríguez, L., Agrawal, S., Aumiller, D., Alva-Manchego, F., and Shardlow, M. 2023. BLESS: Benchmarking Large Language Models on Sentence Simplification. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. Singapore. Association for Computational Linguistics.
Download Paper

Document-level Text Simplification with Coherence Evaluation.

Published in In Proceedings of the Second Workshop on Text Simplification, Accessibility, and Readability (TSAR-2023). Varna, Bulgaria. Recent Advances in Natural Language Processing (RANLP 2023)., 2023

Recommended citation: Vásquez-Rodríguez, L., Shardlow, M., Przybyła, P., and Ananiadou, S. 2023. Document-level Text Simplification with Coherence Evaluation. In Proceedings of the Second Workshop on Text Simplification, Accessibility, and Readability (TSAR-2023). Varna, Bulgaria. Recent Advances in Natural Language Processing (RANLP 2023).
Download Paper

A Benchmark for Neural Readability Assessment of Texts in Spanish

Published in In Proceedings of the Workshop on Text Simplification, Accessibility, and Readability (TSAR-2022), Abu Dhabi, United Arab Emirates (Virtual)., 2022

We release a new benchmark for Automated Readability Assessment (ARA) of texts in Spanish. We combined existing corpora with suitable texts collected from the Web, thus creating the largest available dataset for ARA of Spanish texts. All data was pre-processed and categorised to allow experimenting with ARA models that make predictions at two (simple and complex) or three (basic, intermediate, and advanced) readability levels, and at two text granularities (paragraphs and sentences). An analysis based on readability indices shows that our proposed datasets groupings are suitable for their designated readability level. We use our benchmark to train neural ARA models based on BERT in zero-shot, few-shot, and cross-lingual settings. Results show that either a monolingual or multilingual pre-trained model can achieve good results when fine-tuned in language-specific data. In addition, all models decrease their performance when predicting three classes instead of two, showing opportunities for the development of better ARA models for Spanish with existing resources.

Recommended citation: Vásquez-Rodríguez, L., Cuenca-Jiménez, P., Morales-Esquivel, S., and Alva-Manchego, F. 2022. A Benchmark for Neural Readability Assessment of Texts in Spanish. In Proceedings of the Workshop on Text Simplification, Accessibility, and Readability (TSAR-2022), pages 188–198, Abu Dhabi, United Arab Emirates (Virtual). Association for Computational Linguistics.
Download Paper | Download Slides

UoM and MMU @ TSAR2022-ST — PromptLS: Prompt Learning for Lexical Simplification.

Published in In Proceedings of the Workshop on Text Simplification, Accessibility, and Readability (TSAR-2022), Abu Dhabi, United Arab Emirates (Virtual)., 2022

We present PromptLS, a method for fine-tuning large pre-trained Language Models (LM) to perform the task of Lexical Simplification. We use a predefined template to attain appropriate replacements for a term, and fine-tune a LM using this template on language specific datasets. We filter candidate lists in post-processing to improve accuracy. We demonstrate that our model can work in a) a zero shot setting (where we only require a pre-trained LM), b) a fine-tuned setting (where language-specific data is required), and c) a multilingual setting (where the model is pre-trained across multiple languages and fine-tuned in an specific language). Experimental results show that, although the zero-shot setting is competitive, its performance is still far from the fine-tuned setting. Also, the multilingual is unsurprisingly worse than the fine-tuned model. Among all TSAR-2022 Shared Task participants, our team was ranked second in Spanish and third in English.

Recommended citation: Vásquez-Rodríguez, L., Nguyen, N., Shardlow, M. and Ananiadou, S.. 2022. UoM and MMU @ TSAR2022-ST — PromptLS: Prompt Learning for Lexical Simplification. In Proceedings of the Workshop on Text Simplification, Accessibility, and Readability (TSAR-2022), pages 218–224, Abu Dhabi, United Arab Emirates (Virtual). Association for Computational Linguistics.
Download Paper | Download Slides

The Role of Text Simplification Operations in Evaluation.

Published in In Current Trends in Text Simplification (CTTS 2021), co-located with SEPLN 2021. (Online)., 2021

Research in Text Simplification (TS) has relied mostly on the Wikipedia-based datasets and the SARI evaluation metric, as the preferred means for creating and evaluating new simplification methods. Previous studies have pointed out the flaws of data evaluation resources, including incorrect alignment of simple/complex sentence pairs, sentences with no simplifications or a dearth in the variety of simplification operations. However, there are no further analyses on the impact of the original data distribution regarding the type of simplification operations performed. In this paper, we set up a systematic benchmark of the most common TS datasets, basing our evaluation on different protocols for split selection (e.g., selection by random or by Monte Carlo). We perform an operation-based investigation, demonstrating in detail the limitations of existing simplification datasets. Further, we make recommendations for future standardised practices in the design, creation and evaluation of TS resources.

Recommended citation: Vásquez-Rodríguez, L., Shardlow, M., Przybyła, P., and Ananiadou, S. (2021). The Role of Text Simplification Operations in Evaluation. In Current Trends in Text Simplification (CTTS 2021), co-located with SEPLN 2021. (Online).
Download Paper | Download Slides

Investigating Text Simplification Evaluation

Published in Evaluation. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021. (Online)., 2021

Modern text simplification (TS) heavily relies on the availability of gold standard data to build machine learning models. However, existing studies show that parallel TS corpora contain inaccurate simplifications and incorrect alignments. Additionally, evaluation is usually performed by using metrics such as BLEU or SARI to compare system output to the gold standard. A major limitation is that these metrics do not match human judgements and the performance on different datasets and linguistic phenomena vary greatly. Furthermore, our research shows that the test and training subsets of parallel datasets differ significantly. In this work, we investigate existing TS corpora, providing new insights that will motivate the improvement of existing state-of-the-art TS evaluation methods. Our contributions include the analysis of TS corpora based on existing modifications used for simplification and an empirical study on TS models performance by using better-distributed datasets. We demonstrate that by improving the distribution of TS datasets, we can build more robust TS models.

Recommended citation: Vásquez-Rodríguez, L., Shardlow, M., Przybyła, P., and Ananiadou, S. (2021). Investigating Text Simplification Evaluation. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021. (Online).
Download Paper | Download Slides

PATHWAYS ANALYZER: Design of a Tool for the Synthetic Assembly of Escherichia Coli K-12 MG1655 Bacteria for Biofuel Production.

Published in BioRxiv. The preprint server for Biology., 2019

The present work aims to design a tool for the transformation of metabolic pathways and the development of path finding algorithms that establish relevant links between compounds that are essential to the biofuel production process. As a result, a catalog of biobricks is created from the analysis of a subset of paths which can be used in the design stage of the synthetic assembly of the E. coli bacteria. The assembly’s structure and functions are characterized according to the pieces used. Finally, new constructions are visualized with the goal of demonstrating and supporting the analysis processes, thus assisting people that work in the field of Synthetic Biology.

Recommended citation: Vásquez-Rodríguez, L., Alvarado, R., Orozco, A. (2019). PATHWAYS ANALYZER: Design of a Tool for the Synthetic Assembly of Escherichia Coli K-12 MG1655 Bacteria for Biofuel Production. BioRxiv.
Download Paper

Laura Vásquez-Rodríguez

Publications

Conference Papers