WIP: fix_llama4_tool_call

[wukaixingxp]

Change the llama4 pythonic template and small fix on the edge case where llama4 model may output <|python_start|> unexpectedly.
For jinja template please see this example:
Given this test data:

{
    "bos_token": "<|begin_of_text|>",
    #"tools_in_user_message": true,
    "add_generation_prompt": true,
 "custom_tools": [
    {
        "name": "get_weather",
        "description": "Get weather info for places",
        "parameters": {
            "type": "dict",
            "required": ["city"],
            "properties": {
                "city": {
                    "type": "string",
                    "description": "The name of the city to get the weather for"
                },
                "metric": {
                    "type": "string",
                    "description": "The metric for weather. Options are: celsius, fahrenheit",
                    "default": "celsius"
                }
            }
        }
    }
],
    "messages": [
{"role": "user", "content": "Who are you?"},
{"role": "assistant", "content": "I am a Llama 4 model",tool_calls:[]},
      {"role": "user", "content": "What is the weather in SF and Seattle?"},
{"role": "assistant", "content": "[get_weather(city=\"San Francisco\"), get_weather(city=\"Seattle\")]",tool_calls:[]},
{"role": "tool", "content": "Sunny and Rainy"},
{"role": "assistant", "content": "SF is Sunny and Seattle is Rainy"},
{"role": "user", "content": "What is the weather in NYC",tool_calls:[]},
{"role": "assistant", "content": "",'tool_calls': [
            {
                'name': 'get_weather',
                'arguments': {
                    'city': 'NYC',
                   'metric': 'fahrenheit',
                }
            },
        ],},
    ]
  }

The jinja template will render the output like this:

<|begin_of_text|><|header_start|>system<|header_end|>

You are a helpful assistant and an expert in function composition. You can answer general questions using your internal knowledge OR invoke functions when necessary. Follow these strict guidelines:

1. FUNCTION CALLS:
- ONLY use functions that are EXPLICITLY listed in the function list below
- If NO functions are listed (empty function list []), respond ONLY with internal knowledge or "I don't have access to [Unavailable service] information"
- If a function is not in the list, respond ONLY with internal knowledge or "I don't have access to [Unavailable service] information"
- If ALL required parameters are present AND the query EXACTLY matches a listed function's purpose: output ONLY the function call(s)
- Use exact format: [func_name1(param1=value1, param2=value2), func_name2(...)]
Examples:
CORRECT: [get_weather(location="Vancouver"), calculate_route(start="Boston", end="New York")] <- Only if get_weather and calculate_route are in function list
INCORRECT: get_weather(location="New York")
INCORRECT: Let me check the weather: [get_weather(location="New York")]
INCORRECT: [get_events(location="Singapore")] <- If function not in list

2. RESPONSE RULES:
- For pure function requests matching a listed function: ONLY output the function call(s)
- For knowledge questions: ONLY output text
- For missing parameters: ONLY request the specific missing parameters
- For unavailable services (not in function list): output ONLY with internal knowledge or "I don't have access to [Unavailable service] information". Do NOT execute a function call.
- If the query asks for information beyond what a listed function provides: output ONLY with internal knowledge about your limitations
- NEVER combine text and function calls in the same response
- NEVER suggest alternative functions when the requested service is unavailable
- NEVER create or invent new functions not listed below

3. STRICT BOUNDARIES:
- ONLY use functions from the list below - no exceptions
- NEVER use a function as an alternative to unavailable information
- NEVER call functions not present in the function list
- NEVER add explanatory text to function calls
- NEVER respond with empty brackets
- Use proper Python/JSON syntax for function calls
- Check the function list carefully before responding

4. TOOL RESPONSE HANDLING:
- When receiving tool responses: provide concise, natural language responses
- Don't repeat tool response verbatim
- Don't add supplementary information

Here is a list of functions in JSON format that you can invoke:
[
    {
        "description": "Get weather info for places",
        "name": "get_weather",
        "parameters": {
            "properties": {
                "city": {
                    "description": "The name of the city to get the weather for",
                    "type": "string"
                },
                "metric": {
                    "default": "celsius",
                    "description": "The metric for weather. Options are: celsius, fahrenheit",
                    "type": "string"
                }
            },
            "required": [
                "city"
            ],
            "type": "dict"
        }
    }
]
<|eot|><|header_start|>user<|header_end|>

Who are you?<|eot|><|header_start|>assistant<|header_end|>

I am a Llama 4 model<|eot|><|header_start|>user<|header_end|>

What is the weather in SF and Seattle?<|eot|><|header_start|>assistant<|header_end|>

[get_weather(city="San Francisco"), get_weather(city="Seattle")]<|eot|><|header_start|>ipython<|header_end|>

{"tool_output": "Sunny and Rainy"}<|eot|><|header_start|>assistant<|header_end|>

SF is Sunny and Seattle is Rainy<|eot|><|header_start|>assistant<|header_end|>

[get_weather(city="NYC", metric="fahrenheit")]<|eot|><|header_start|>assistant<|header_end|>

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[yeqcharlotte]

great thank you! please fix the long lines.

and do could you get the results from llama-stack eval?
please also double check unit tests on vllm side pytest -s -vv tests/tool_use --models llama4 --extended

[yeqcharlotte]

ah i see it's confirmed we should switched back to eot now?

[yeqcharlotte]

could you add some unit tests for these logic?