Benefits:

Improves diversity

Reduces repetitive outputs

Balances creativity and quality

Top-p is often tuned together with temperature.

10. What is Max Tokens?

Max Tokens defines the maximum number of tokens the model is allowed to generate in its response.

Example:

Max Tokens = 100

The response stops after generating up to 100 output tokens, even if the answer could be longer.

This helps control:

Response length

Latency

Cost

11. What is Latency?

Latency is the time taken by the model to generate a response after receiving a request.

Factors affecting latency:

Model size

Prompt length

Context window

Hardware

Network

Retrieval time (for RAG)

12. What is Inference Cost?

Inference cost is the cost of running an LLM for generating responses.

It depends on:

Number of input tokens

Number of output tokens

Model size

Number of API requests

Reducing unnecessary tokens and optimizing prompts can significantly lower costs.

13. Common LLM Interview Questions

What is an LLM?

How are LLMs trained?

What is pretraining?

What is fine-tuning?

What is RLHF?

What are tokens?

What are parameters?

What is inference?

What is a context window?

What is temperature?

What is top-p sampling?

What is inference cost?

🎯 Interview Tip

For LLM questions, use this simple structure:

1. Define the concept.

2. Explain how it works.

3. Give a practical example.

4. Mention a real-world use case.

5. Highlight benefits and limitations.

This approach makes your answers clear, structured, and interview-ready.

➡️ Double Tap ❤️ For More