Feature engineering is a vital stage in machine learning pipelines that greatly affects the performance, interpretability, and general efficacy of models. Filter, wrapper, and embedded techniques are common ways to choose and change features, but they often need manual heuristics and subject knowledge. They also don't work well in environments with a lot of dimensions and complexity. Recent studies have investigated automated methods that make use of large language models and reinforcement learning in order to overcome these constraints. A comprehensive and critically synthesized survey of state of the art works covering RL-based feature selection, RL-driven feature generation, and LLM-guided feature optimization is presented in this paper.Three main paradigms of methodology are identified. In the first, feature selection is framed as a cooperative or guided decision making problem using interactive and multi-agent reinforcement learning techniques. These techniques allocate agents to features and maximize long-term rewards according to domain-specific significance, redundancy, or model accuracy. Combinatorial Multi-Armed Bandits (CMAB), a computationally lightweight alternative that provides scalable and effective feature selection with little learning overhead, is part of the second paradigm cite{li2022bandit}. For the third group, LLMs are used to either learn successful reward functions or make new features. They do this by using reasoning-based prompts, external knowledge bases, and prototypical alignment. This work also address open challenges in bias control, compute overhead, and generalization to unseen domains as well as underexplored gaps including the need of hybrid frameworks combining RL's exploration efficiency with LLMs's semantic reasoning.