Large language models have demonstrated remarkable capabilities across diverse natural language tasks, yet controlling their output characteristics remains challenging. We present HelpSteer Transformer, an attribute-conditioned language model architecture designed for training on the HelpSteer dataset. The model incorporates modern architectural innovations including Rotary Position Embeddings (RoPE), SwiGLU activation functions, and RMSNorm, enabling fine-grained control over five response attributes: helpfulness, correctness, coherence, complexity, and verbosity.The model contains approximately 60 million parameters across eight transformer layers and is designed for efficient scaling while maintaining high-quality text generation. An explicit attribute conditioning mechanism integrates user preferences directly into the generation process, enabling dynamic control of outputs without requiring separate fine-tuning for different attribute combinations. Architectural analysis and preliminary experiments indicate competitive performance relative to larger baseline models, while maintaining lower computational cost. This work highlights the effectiveness of architectural conditioning for controllable and efficient language model design.