• Python »
  • 3.12.2 Documentation »
  • The Python Standard Library »
  • Networking and Interprocess Communication »
  • asyncio — Asynchronous I/O »
  • Coroutines and Tasks
  • Theme Auto Light Dark |

Coroutines and Tasks ¶

This section outlines high-level asyncio APIs to work with coroutines and Tasks.

Coroutines ¶

Source code: Lib/asyncio/coroutines.py

Coroutines declared with the async/await syntax is the preferred way of writing asyncio applications. For example, the following snippet of code prints “hello”, waits 1 second, and then prints “world”:

Note that simply calling a coroutine will not schedule it to be executed:

To actually run a coroutine, asyncio provides the following mechanisms:

The asyncio.run() function to run the top-level entry point “main()” function (see the above example.)

Awaiting on a coroutine. The following snippet of code will print “hello” after waiting for 1 second, and then print “world” after waiting for another 2 seconds:

Expected output:

The asyncio.create_task() function to run coroutines concurrently as asyncio Tasks .

Let’s modify the above example and run two say_after coroutines concurrently :

Note that expected output now shows that the snippet runs 1 second faster than before:

The asyncio.TaskGroup class provides a more modern alternative to create_task() . Using this API, the last example becomes:

The timing and output should be the same as for the previous version.

New in version 3.11: asyncio.TaskGroup .

Awaitables ¶

We say that an object is an awaitable object if it can be used in an await expression. Many asyncio APIs are designed to accept awaitables.

There are three main types of awaitable objects: coroutines , Tasks , and Futures .

Python coroutines are awaitables and therefore can be awaited from other coroutines:

In this documentation the term “coroutine” can be used for two closely related concepts:

a coroutine function : an async def function;

a coroutine object : an object returned by calling a coroutine function .

Tasks are used to schedule coroutines concurrently .

When a coroutine is wrapped into a Task with functions like asyncio.create_task() the coroutine is automatically scheduled to run soon:

A Future is a special low-level awaitable object that represents an eventual result of an asynchronous operation.

When a Future object is awaited it means that the coroutine will wait until the Future is resolved in some other place.

Future objects in asyncio are needed to allow callback-based code to be used with async/await.

Normally there is no need to create Future objects at the application level code.

Future objects, sometimes exposed by libraries and some asyncio APIs, can be awaited:

A good example of a low-level function that returns a Future object is loop.run_in_executor() .

Creating Tasks ¶

Source code: Lib/asyncio/tasks.py

Wrap the coro coroutine into a Task and schedule its execution. Return the Task object.

If name is not None , it is set as the name of the task using Task.set_name() .

An optional keyword-only context argument allows specifying a custom contextvars.Context for the coro to run in. The current context copy is created when no context is provided.

The task is executed in the loop returned by get_running_loop() , RuntimeError is raised if there is no running loop in current thread.

asyncio.TaskGroup.create_task() is a new alternative leveraging structural concurrency; it allows for waiting for a group of related tasks with strong safety guarantees.

Save a reference to the result of this function, to avoid a task disappearing mid-execution. The event loop only keeps weak references to tasks. A task that isn’t referenced elsewhere may get garbage collected at any time, even before it’s done. For reliable “fire-and-forget” background tasks, gather them in a collection:

New in version 3.7.

Changed in version 3.8: Added the name parameter.

Changed in version 3.11: Added the context parameter.

Task Cancellation ¶

Tasks can easily and safely be cancelled. When a task is cancelled, asyncio.CancelledError will be raised in the task at the next opportunity.

It is recommended that coroutines use try/finally blocks to robustly perform clean-up logic. In case asyncio.CancelledError is explicitly caught, it should generally be propagated when clean-up is complete. asyncio.CancelledError directly subclasses BaseException so most code will not need to be aware of it.

The asyncio components that enable structured concurrency, like asyncio.TaskGroup and asyncio.timeout() , are implemented using cancellation internally and might misbehave if a coroutine swallows asyncio.CancelledError . Similarly, user code should not generally call uncancel . However, in cases when suppressing asyncio.CancelledError is truly desired, it is necessary to also call uncancel() to completely remove the cancellation state.

Task Groups ¶

Task groups combine a task creation API with a convenient and reliable way to wait for all tasks in the group to finish.

An asynchronous context manager holding a group of tasks. Tasks can be added to the group using create_task() . All tasks are awaited when the context manager exits.

New in version 3.11.

Create a task in this task group. The signature matches that of asyncio.create_task() .

The async with statement will wait for all tasks in the group to finish. While waiting, new tasks may still be added to the group (for example, by passing tg into one of the coroutines and calling tg.create_task() in that coroutine). Once the last task has finished and the async with block is exited, no new tasks may be added to the group.

The first time any of the tasks belonging to the group fails with an exception other than asyncio.CancelledError , the remaining tasks in the group are cancelled. No further tasks can then be added to the group. At this point, if the body of the async with statement is still active (i.e., __aexit__() hasn’t been called yet), the task directly containing the async with statement is also cancelled. The resulting asyncio.CancelledError will interrupt an await , but it will not bubble out of the containing async with statement.

Once all tasks have finished, if any tasks have failed with an exception other than asyncio.CancelledError , those exceptions are combined in an ExceptionGroup or BaseExceptionGroup (as appropriate; see their documentation) which is then raised.

Two base exceptions are treated specially: If any task fails with KeyboardInterrupt or SystemExit , the task group still cancels the remaining tasks and waits for them, but then the initial KeyboardInterrupt or SystemExit is re-raised instead of ExceptionGroup or BaseExceptionGroup .

If the body of the async with statement exits with an exception (so __aexit__() is called with an exception set), this is treated the same as if one of the tasks failed: the remaining tasks are cancelled and then waited for, and non-cancellation exceptions are grouped into an exception group and raised. The exception passed into __aexit__() , unless it is asyncio.CancelledError , is also included in the exception group. The same special case is made for KeyboardInterrupt and SystemExit as in the previous paragraph.

Block for delay seconds.

If result is provided, it is returned to the caller when the coroutine completes.

sleep() always suspends the current task, allowing other tasks to run.

Setting the delay to 0 provides an optimized path to allow other tasks to run. This can be used by long-running functions to avoid blocking the event loop for the full duration of the function call.

Example of coroutine displaying the current date every second for 5 seconds:

Changed in version 3.10: Removed the loop parameter.

Running Tasks Concurrently ¶

Run awaitable objects in the aws sequence concurrently .

If any awaitable in aws is a coroutine, it is automatically scheduled as a Task.

If all awaitables are completed successfully, the result is an aggregate list of returned values. The order of result values corresponds to the order of awaitables in aws .

If return_exceptions is False (default), the first raised exception is immediately propagated to the task that awaits on gather() . Other awaitables in the aws sequence won’t be cancelled and will continue to run.

If return_exceptions is True , exceptions are treated the same as successful results, and aggregated in the result list.

If gather() is cancelled , all submitted awaitables (that have not completed yet) are also cancelled .

If any Task or Future from the aws sequence is cancelled , it is treated as if it raised CancelledError – the gather() call is not cancelled in this case. This is to prevent the cancellation of one submitted Task/Future to cause other Tasks/Futures to be cancelled.

A new alternative to create and run tasks concurrently and wait for their completion is asyncio.TaskGroup . TaskGroup provides stronger safety guarantees than gather for scheduling a nesting of subtasks: if a task (or a subtask, a task scheduled by a task) raises an exception, TaskGroup will, while gather will not, cancel the remaining scheduled tasks).

If return_exceptions is False, cancelling gather() after it has been marked done won’t cancel any submitted awaitables. For instance, gather can be marked done after propagating an exception to the caller, therefore, calling gather.cancel() after catching an exception (raised by one of the awaitables) from gather won’t cancel any other awaitables.

Changed in version 3.7: If the gather itself is cancelled, the cancellation is propagated regardless of return_exceptions .

Deprecated since version 3.10: Deprecation warning is emitted if no positional arguments are provided or not all positional arguments are Future-like objects and there is no running event loop.

Eager Task Factory ¶

A task factory for eager task execution.

When using this factory (via loop.set_task_factory(asyncio.eager_task_factory) ), coroutines begin execution synchronously during Task construction. Tasks are only scheduled on the event loop if they block. This can be a performance improvement as the overhead of loop scheduling is avoided for coroutines that complete synchronously.

A common example where this is beneficial is coroutines which employ caching or memoization to avoid actual I/O when possible.

Immediate execution of the coroutine is a semantic change. If the coroutine returns or raises, the task is never scheduled to the event loop. If the coroutine execution blocks, the task is scheduled to the event loop. This change may introduce behavior changes to existing applications. For example, the application’s task execution order is likely to change.

New in version 3.12.

Create an eager task factory, similar to eager_task_factory() , using the provided custom_task_constructor when creating a new task instead of the default Task .

custom_task_constructor must be a callable with the signature matching the signature of Task.__init__ . The callable must return a asyncio.Task -compatible object.

This function returns a callable intended to be used as a task factory of an event loop via loop.set_task_factory(factory) ).

Shielding From Cancellation ¶

Protect an awaitable object from being cancelled .

If aw is a coroutine it is automatically scheduled as a Task.

The statement:

is equivalent to:

except that if the coroutine containing it is cancelled, the Task running in something() is not cancelled. From the point of view of something() , the cancellation did not happen. Although its caller is still cancelled, so the “await” expression still raises a CancelledError .

If something() is cancelled by other means (i.e. from within itself) that would also cancel shield() .

If it is desired to completely ignore cancellation (not recommended) the shield() function should be combined with a try/except clause, as follows:

Save a reference to tasks passed to this function, to avoid a task disappearing mid-execution. The event loop only keeps weak references to tasks. A task that isn’t referenced elsewhere may get garbage collected at any time, even before it’s done.

Deprecated since version 3.10: Deprecation warning is emitted if aw is not Future-like object and there is no running event loop.

Return an asynchronous context manager that can be used to limit the amount of time spent waiting on something.

delay can either be None , or a float/int number of seconds to wait. If delay is None , no time limit will be applied; this can be useful if the delay is unknown when the context manager is created.

In either case, the context manager can be rescheduled after creation using Timeout.reschedule() .

If long_running_task takes more than 10 seconds to complete, the context manager will cancel the current task and handle the resulting asyncio.CancelledError internally, transforming it into a TimeoutError which can be caught and handled.

The asyncio.timeout() context manager is what transforms the asyncio.CancelledError into a TimeoutError , which means the TimeoutError can only be caught outside of the context manager.

Example of catching TimeoutError :

The context manager produced by asyncio.timeout() can be rescheduled to a different deadline and inspected.

An asynchronous context manager for cancelling overdue coroutines.

when should be an absolute time at which the context should time out, as measured by the event loop’s clock:

If when is None , the timeout will never trigger.

If when < loop.time() , the timeout will trigger on the next iteration of the event loop.

when ( ) → float | None ¶ Return the current deadline, or None if the current deadline is not set. reschedule ( when : float | None ) ¶ Reschedule the timeout. expired ( ) → bool ¶ Return whether the context manager has exceeded its deadline (expired).

Timeout context managers can be safely nested.

Similar to asyncio.timeout() , except when is the absolute time to stop waiting, or None .

Wait for the aw awaitable to complete with a timeout.

timeout can either be None or a float or int number of seconds to wait for. If timeout is None , block until the future completes.

If a timeout occurs, it cancels the task and raises TimeoutError .

To avoid the task cancellation , wrap it in shield() .

The function will wait until the future is actually cancelled, so the total wait time may exceed the timeout . If an exception happens during cancellation, it is propagated.

If the wait is cancelled, the future aw is also cancelled.

Changed in version 3.7: When aw is cancelled due to a timeout, wait_for waits for aw to be cancelled. Previously, it raised TimeoutError immediately.

Changed in version 3.11: Raises TimeoutError instead of asyncio.TimeoutError .

Waiting Primitives ¶

Run Future and Task instances in the aws iterable concurrently and block until the condition specified by return_when .

The aws iterable must not be empty.

Returns two sets of Tasks/Futures: (done, pending) .

timeout (a float or int), if specified, can be used to control the maximum number of seconds to wait before returning.

Note that this function does not raise TimeoutError . Futures or Tasks that aren’t done when the timeout occurs are simply returned in the second set.

return_when indicates when this function should return. It must be one of the following constants:

Unlike wait_for() , wait() does not cancel the futures when a timeout occurs.

Changed in version 3.11: Passing coroutine objects to wait() directly is forbidden.

Changed in version 3.12: Added support for generators yielding tasks.

Run awaitable objects in the aws iterable concurrently. Return an iterator of coroutines. Each coroutine returned can be awaited to get the earliest next result from the iterable of the remaining awaitables.

Raises TimeoutError if the timeout occurs before all Futures are done.

Deprecated since version 3.10: Deprecation warning is emitted if not all awaitable objects in the aws iterable are Future-like objects and there is no running event loop.

Running in Threads ¶

Asynchronously run function func in a separate thread.

Any *args and **kwargs supplied for this function are directly passed to func . Also, the current contextvars.Context is propagated, allowing context variables from the event loop thread to be accessed in the separate thread.

Return a coroutine that can be awaited to get the eventual result of func .

This coroutine function is primarily intended to be used for executing IO-bound functions/methods that would otherwise block the event loop if they were run in the main thread. For example:

Directly calling blocking_io() in any coroutine would block the event loop for its duration, resulting in an additional 1 second of run time. Instead, by using asyncio.to_thread() , we can run it in a separate thread without blocking the event loop.

Due to the GIL , asyncio.to_thread() can typically only be used to make IO-bound functions non-blocking. However, for extension modules that release the GIL or alternative Python implementations that don’t have one, asyncio.to_thread() can also be used for CPU-bound functions.

New in version 3.9.

Scheduling From Other Threads ¶

Submit a coroutine to the given event loop. Thread-safe.

Return a concurrent.futures.Future to wait for the result from another OS thread.

This function is meant to be called from a different OS thread than the one where the event loop is running. Example:

If an exception is raised in the coroutine, the returned Future will be notified. It can also be used to cancel the task in the event loop:

See the concurrency and multithreading section of the documentation.

Unlike other asyncio functions this function requires the loop argument to be passed explicitly.

New in version 3.5.1.

Introspection ¶

Return the currently running Task instance, or None if no task is running.

If loop is None get_running_loop() is used to get the current loop.

Return a set of not yet finished Task objects run by the loop.

If loop is None , get_running_loop() is used for getting current loop.

Return True if obj is a coroutine object.

New in version 3.4.

Task Object ¶

A Future-like object that runs a Python coroutine . Not thread-safe.

Tasks are used to run coroutines in event loops. If a coroutine awaits on a Future, the Task suspends the execution of the coroutine and waits for the completion of the Future. When the Future is done , the execution of the wrapped coroutine resumes.

Event loops use cooperative scheduling: an event loop runs one Task at a time. While a Task awaits for the completion of a Future, the event loop runs other Tasks, callbacks, or performs IO operations.

Use the high-level asyncio.create_task() function to create Tasks, or the low-level loop.create_task() or ensure_future() functions. Manual instantiation of Tasks is discouraged.

To cancel a running Task use the cancel() method. Calling it will cause the Task to throw a CancelledError exception into the wrapped coroutine. If a coroutine is awaiting on a Future object during cancellation, the Future object will be cancelled.

cancelled() can be used to check if the Task was cancelled. The method returns True if the wrapped coroutine did not suppress the CancelledError exception and was actually cancelled.

asyncio.Task inherits from Future all of its APIs except Future.set_result() and Future.set_exception() .

An optional keyword-only context argument allows specifying a custom contextvars.Context for the coro to run in. If no context is provided, the Task copies the current context and later runs its coroutine in the copied context.

An optional keyword-only eager_start argument allows eagerly starting the execution of the asyncio.Task at task creation time. If set to True and the event loop is running, the task will start executing the coroutine immediately, until the first time the coroutine blocks. If the coroutine returns or raises without blocking, the task will be finished eagerly and will skip scheduling to the event loop.

Changed in version 3.7: Added support for the contextvars module.

Deprecated since version 3.10: Deprecation warning is emitted if loop is not specified and there is no running event loop.

Changed in version 3.12: Added the eager_start parameter.

Return True if the Task is done .

A Task is done when the wrapped coroutine either returned a value, raised an exception, or the Task was cancelled.

Return the result of the Task.

If the Task is done , the result of the wrapped coroutine is returned (or if the coroutine raised an exception, that exception is re-raised.)

If the Task has been cancelled , this method raises a CancelledError exception.

If the Task’s result isn’t yet available, this method raises a InvalidStateError exception.

Return the exception of the Task.

If the wrapped coroutine raised an exception that exception is returned. If the wrapped coroutine returned normally this method returns None .

If the Task isn’t done yet, this method raises an InvalidStateError exception.

Add a callback to be run when the Task is done .

This method should only be used in low-level callback-based code.

See the documentation of Future.add_done_callback() for more details.

Remove callback from the callbacks list.

See the documentation of Future.remove_done_callback() for more details.

Return the list of stack frames for this Task.

If the wrapped coroutine is not done, this returns the stack where it is suspended. If the coroutine has completed successfully or was cancelled, this returns an empty list. If the coroutine was terminated by an exception, this returns the list of traceback frames.

The frames are always ordered from oldest to newest.

Only one stack frame is returned for a suspended coroutine.

The optional limit argument sets the maximum number of frames to return; by default all available frames are returned. The ordering of the returned list differs depending on whether a stack or a traceback is returned: the newest frames of a stack are returned, but the oldest frames of a traceback are returned. (This matches the behavior of the traceback module.)

Print the stack or traceback for this Task.

This produces output similar to that of the traceback module for the frames retrieved by get_stack() .

The limit argument is passed to get_stack() directly.

The file argument is an I/O stream to which the output is written; by default output is written to sys.stdout .

Return the coroutine object wrapped by the Task .

This will return None for Tasks which have already completed eagerly. See the Eager Task Factory .

New in version 3.8.

Changed in version 3.12: Newly added eager task execution means result may be None .

Return the contextvars.Context object associated with the task.

Return the name of the Task.

If no name has been explicitly assigned to the Task, the default asyncio Task implementation generates a default name during instantiation.

Set the name of the Task.

The value argument can be any object, which is then converted to a string.

In the default Task implementation, the name will be visible in the repr() output of a task object.

Request the Task to be cancelled.

This arranges for a CancelledError exception to be thrown into the wrapped coroutine on the next cycle of the event loop.

The coroutine then has a chance to clean up or even deny the request by suppressing the exception with a try … … except CancelledError … finally block. Therefore, unlike Future.cancel() , Task.cancel() does not guarantee that the Task will be cancelled, although suppressing cancellation completely is not common and is actively discouraged. Should the coroutine nevertheless decide to suppress the cancellation, it needs to call Task.uncancel() in addition to catching the exception.

Changed in version 3.9: Added the msg parameter.

Changed in version 3.11: The msg parameter is propagated from cancelled task to its awaiter.

The following example illustrates how coroutines can intercept the cancellation request:

Return True if the Task is cancelled .

The Task is cancelled when the cancellation was requested with cancel() and the wrapped coroutine propagated the CancelledError exception thrown into it.

Decrement the count of cancellation requests to this Task.

Returns the remaining number of cancellation requests.

Note that once execution of a cancelled task completed, further calls to uncancel() are ineffective.

This method is used by asyncio’s internals and isn’t expected to be used by end-user code. In particular, if a Task gets successfully uncancelled, this allows for elements of structured concurrency like Task Groups and asyncio.timeout() to continue running, isolating cancellation to the respective structured block. For example:

While the block with make_request() and make_another_request() might get cancelled due to the timeout, unrelated_code() should continue running even in case of the timeout. This is implemented with uncancel() . TaskGroup context managers use uncancel() in a similar fashion.

If end-user code is, for some reason, suppresing cancellation by catching CancelledError , it needs to call this method to remove the cancellation state.

Return the number of pending cancellation requests to this Task, i.e., the number of calls to cancel() less the number of uncancel() calls.

Note that if this number is greater than zero but the Task is still executing, cancelled() will still return False . This is because this number can be lowered by calling uncancel() , which can lead to the task not being cancelled after all if the cancellation requests go down to zero.

This method is used by asyncio’s internals and isn’t expected to be used by end-user code. See uncancel() for more details.

Table of Contents

  • Creating Tasks
  • Task Cancellation
  • Task Groups
  • Running Tasks Concurrently
  • Eager Task Factory
  • Shielding From Cancellation
  • Waiting Primitives
  • Running in Threads
  • Scheduling From Other Threads
  • Introspection
  • Task Object

Previous topic

  • Report a Bug
  • Show Source

Home » Python Concurrency » Python asyncio.create_task()

Python asyncio.create_task()

Summary : in this tutorial, you’ll learn how to use asyncio.create_task() function to run multiple tasks concurrently.

Simulating a long-running operation

To simulate a long-running operation, you can use the sleep() coroutine of the asyncio package. The sleep() function delays a specified number of the second:

Because sleep() is a coroutine, you need to use the await keyword. For example, the following uses the sleep() coroutine to simulate an API call:

The call_api() is a coroutine. It displays a message, pauses a specified number of seconds (default to three seconds), and returns a result.

The following program uses the call_api() twice and measures the time it takes to complete:

How it works (focusing on the main() coroutine):

First, start a timer to measure the time using the perf_counter() function of the time module:

Second, call the call_api() coroutine and display the result:

Third, call the call_api() a second time:

Finally, show the time the program takes to complete:

Because each call_api() takes three seconds, and calling it twice takes six seconds.

In this example, we call a coroutine directly and don’t put it on the event loop to run. Instead, we get a coroutine object and use the await keyword to execute it and get a result.

The following picture illustrates the execution flow of the program:

In other words, we use async and await to write asynchronous code but can’t run it concurrently. To run multiple operations concurrently, we’ll need to use something called tasks.

Introduction to Python tasks

  • A task is a wrapper of a coroutine that schedules the coroutine to run on the event loop as soon as possible.

The scheduling and execution occur in a non-blocking manner. In other words, you can create a task and execute other code instantly while the task is running.

Notice that the task is different from the await keyword that blocks the entire coroutine until the operation completes with a result.

It’s important that you can create multiple tasks and schedule them to run instantly on the event loop at the same time.

To create a task, you pass a coroutine to the create_task() function of the asyncio package. The create_task() function returns a Task object.

The following program illustrates how to create two tasks that schedule and execute the call_api() coroutine:

How it works.

First, start a timer:

Next, create a task and schedule it to run on the event loop immediately:

Then, create another task and schedule it to run on the event loop immediately:

After that, wait for the tasks to be completed:

It’s important to use the await keyword to wait for the tasks at some point in the program.

If we did not use the await keyword, Python would schedule the task to run but stopped it when the asyncio.run() shutdown the event loop.

Finally, show the time it takes to complete the main() function:

By using the create_task() function, the program is much faster. The more tasks you run, the faster it is.

Running other tasks while waiting

When the call_api is running, you can run other tasks. For example, the following program displays a message every second while waiting for the call_api tasks:

The following picture illustrates the execution flow:

  • Use the create_task() function of the asyncio library to create a task.
  • Use the await keyword with the task at some point in the program so that the task can be completed before the event loop is closed by the asyncio.run() function.

Python: Using the “yield” keyword with async/await (3 examples)

In Python, the yield keyword is used to create a generator, which is a special kind of iterator that can be paused and resumed. In Python 3.6 and above, you can use the yield keyword inside an async function to create an asynchronous generator, which is an iterator that can also perform asynchronous operations.

This succinct, code-centric article will walk you through several examples of using yield with async/await in modern Python. Without any further ado, let’s get started.

A simple async generator that yields numbers (basic)

In this example, we’ll create a simple async generator that yields numbers from 0 to n with a delay of one second:

The code will print 5, 4, 3, 2, 1, and 0 with a one-second interval between each print:

Using “yield” with an asyncio queue (intermediate)

In this example, we’ll use the yield keyword with an asynchronous queue to create an async generator that produces Fibonacci numbers:

The code will generate Fibonacci numbers up to 89. Each number will show up after a one-second delay:

You can find more details about async queues in this article: Python asyncio.Queue class .

Async generator producer-consumer pattern with queue and sentinel (advanced)

Note : The word “sentinel” here means a guard or a watchman. It is used to describe a value that acts as a signal or a marker for the end of a data stream or a loop. A sentinel value is usually chosen to be different from any valid data value so that it can be easily recognized and handled.

This example demonstrates the way to use an async generator to implement a producer-consumer pattern, where one coroutine produces data and another coroutine consumes it. The producer coroutine uses an async queue to store the data, and the consumer coroutine uses an async for loop to get the data from the queue. The producer coroutine also uses a sentinel value to signal the end of the data stream:

Output (you’ll notice a short delay before each print):

Final words

We’ve examined some practical examples of using async generators, which can produce a stream of values asynchronously. You can make use of them to improve the performance and responsiveness of your programs. This tutorial ends here. Happy coding & have fun with modern Python programming!

Previous Article: Python SyntaxError: 'await' outside async function

Series: Python Asynchronous Programming Tutorials

Related Articles

  • Understanding ‘Never’ type in Python 3.11+ (5 examples)
  • Python: 3 Ways to Retrieve City/Country from IP Address
  • Using Type Aliases in Python: A Practical Guide (with Examples)
  • Python: Defining distinct types using NewType class
  • Using Optional Type in Python (explained with examples)
  • Python: How to Override Methods in Classes
  • Python: Define Generic Types for Lists of Nested Dictionaries
  • Python: Defining type for a list that can contain both numbers and strings
  • Using TypeGuard in Python (Python 3.10+)
  • Python: Using ‘NoReturn’ type with functions
  • Type Casting in Python: The Ultimate Guide (with Examples)
  • Python: Using type hints with class methods and properties

guest

Search tutorials, examples, and resources

  • PHP programming
  • Symfony & Doctrine
  • Laravel & Eloquent
  • Tailwind CSS
  • Sequelize.js
  • Mongoose.js

cloudfit-public-docs

Python asyncio part 2 – awaitables, tasks, and futures.

Having already covered the basic concepts in Python Asyncio Part 1 – Basic Concepts and Patterns , in this part of the series I will be going into more depth on the actual syntax used when employing this library in Python code. Many of the examples used here are based on code we have actually used as part of BBC R&D’s cloudfit project .

Writing Asynchronous Code

The most basic tool in the tool kit of an asynchronous programmer in Python is the new keyword async def , which is used to declare an asynchronous coroutine function in the same way that def is used to define a normal synchronous function.

TERMINOLOGY: In this article I will refer to async def as a keyword, and in future articles I will refer to async for and async with as keywords. Strictly speaking this isn’t true. In fact async is a keyword and so is def , but since you can’t use async by itself, only in combination with another keyword I think it’s much more convenient and less confusing to think of async def as a single keyword that happens to have a space in the middle of it. It certainly behaves like one in terms of language usage.

So for example:

In the above example we define a coroutine function example_coroutine_function and an ordinary function example_function . The code block that forms the body of the definition is slightly different in the two cases. The code block for example_function is ordinary synchronous Python, whilst the code-block for example_coroutine_function is asynchronous Python.

IMPORTANT!: Asynchronous Python code can only be included inside a suitable context that allows it, which almost always means inside a coroutine function defined using async def . There’s one other context where asynchronous code is allowed which we will cover in the next article. Asynchronous Python code can use any of the Python keywords, structures, etc… allowed in ordinary Python. Nothing is disallowed (although some things may be discouraged, see later). There are several new keywords which can only be used inside asynchronous code: await , async with and async for . Note that async def is not one of the keywords reserved for use in asynchronous code. It can be used anywhere were def can be used, though its effect is slightly different.

A declaration of a coroutine function using async def looks deceptively similar to the declaration of an ordinary function using def . Most of the time writing one is pretty similar, however there are some key differences, which are very important for asynchronous programming:

means that example_function is now a callable object which takes three parameters. When you invoke it like so:

this causes the function code to be run immediately as a subroutine call, and its return value to be assigned to r .

means that example_coroutine_function is now a callable object which takes three parameters. When you invoke it like so:

this does not cause the function code block to be run. Instead an object of class Coroutine is created, and is assigned to r . To make the code block actually run you need to make use of one of the facilities that asyncio provides for running a coroutine. Most commonly this is the await keyword. The function asyncio.gather is used in an example below. Other examples can be found in the python docs . See for example wait .

TERMINOLOGY: It’s pretty common for people to be sloppy in their terminology and use the word “coroutine” to refer to any of three things: The code block of asynchronous code inside an async def statement. The callable object that the async def statement creates. The object of class Coroutine that is returned by the callable object when it is called. In this series I will try to keep it clear which of these I’m talking about at any particular point. In particular I will usually say “coroutine object” for an object of class Coroutine , and “coroutine function” for the callable that returns it. When I need to refer to the code block specifically (which is not often) I will refer to it as a “code block inside an async def statement which defines a coroutine function”.
TYPING NOTE: If you are using the typing library then the declaration of coroutine functions can be a little confusing at times. async def example_coroutine_function ( a : A , b : B ) -> C : ... defines example_coroutine_function as a callable that takes two parameters of types A and B and returns an object of type Coroutine[Any, Any, C] . It’s pretty rare that you’ll need to refer to this return type explicitly. If you’re curious about the two Any type parameters in the above definition they’re related to the way that the event loop works. The first type parameter actually indicates the type of the values that the coroutine will pass to the event loop whenever it yields, whilst the second represents the type of the values that the event loop will pass back to the coroutine whenever the it is reawakened. In practice the actual types of these objects are determined by the internal machinery of the event loop’s implementation, and should never need to be referred to explicitly in client code unless you are writing your own event loop implementation (which is a pretty advanced topic way beyond the scope of these articles).

The await Keyword and Awaitables

One of the new keywords added to the language to support asyncio is await . This keyword is, in many ways, the very core of asynchronous code. It can only be used inside asynchronous code blocks (ie. in the code block of an async def statement defining a coroutine function), and it is used as an expression which takes a single parameter and returns a value.

is a valid Python statement which will perform the await action on the object a and return a value which will be assigned to r . Exactly what will happen when this await statement is executed will depend upon what the object a is.

A coroutine object is “awaitable” (it can be used in an await statement). Recall that when you are executing asynchronous code you are always doing so in the context of a “Task”, which is an object maintained by the Event Loop, and that each Task has its own call stack. The first time a Coroutine object is awaited the code block inside its definition is executed in the current Task, with its new code context added to the top of the call stack for this Task, just like a normal function call. When the code block reaches its end (or otherwise returns) then execution moves back to the await statement that called it. The return value of the await statement is the value returned by the code block. If a Coroutine object is awaited a second time this raises an exception. In this way you can think of awaiting a Coroutine object as being very much like calling a function, with the notable difference that the Coroutine object’s code block can contain asynchronous code, and so can pause the current task during running, which a function’s code block cannot.

In fact there are three types of objects that are awaitable:

  • A Coroutine object. When awaited it will execute the code-block of the coroutine in the current Task. The await statement will return the value returned by the code block.
  • Any object of class asyncio.Future which when awaited causes the current Task to be paused until a specific condition occurs (see next section).
  • An object which implements the magic method __await__ , in which case what happens when it is awaited is defined by that method.

That last one is there so that writers of libraries can create their own new classes of objects which are awaitable and do something special when awaited. It’s usually a good idea to make your custom awaitable objects either behave like a Coroutine object or like a Future object, and document which in the class’s doc strings. Making custom awaitable classes like this is a somewhat more advanced topic, though one that may come up when writing asyncio wrappers for synchronous io libraries, for example.

TYPING NOTE: If you are using typing then there is an abstract class Awaitable which is generic, so that Awaitable[R] for some type R means “anything which is awaitable, and when used in an await statement will return something of type R ”.

One of the most important points to get across is that the currently executing Task cannot be paused by any means other than awaiting a future (or a custom awaitable object that behaves like one). And that is something which can only happen inside asynchronous code. So any await statement might cause your current task to pause, but is not guaranteed to. Conversely any statement which is not an await statement (or an async for or async with under certain circumstances which will be explained in the next post) cannot cause your current Task to be paused.

This means that the traditional multithreaded code problems of data races where different threads of execution both alter the same value are severely reduced in asynchronous code, but not entirely eliminated. In particular for the purposes of data shared between Tasks on the same event loop all synchronous code can be considered “atomic”. To illustrate what this means consider the following code:

then even though both fetcher and monitor access the global variable vals they do so in two tasks that are running in the same event loop. For this reason it is not possible for the print statement in monitor to run unless fetcher is currently asleep waiting for io. This means that it is not possible for the length of vals to be printed whilst the for loop is only part-way through running. So if the get_some_values_from_io always returns 10 values at a time (for example) then the printed length of vals will always be a multiple of ten. It is simply not possible for the print statement to execute at a time when vals has a non-multiple of ten length.

On the other hand if there was an await statement inside the for loop this would no longer be guaranteed.

NOTE: Note that the create_task calls above are redundant. The body of main could be reduced to await asyncio.gather(fetcher(), monitor()) .

A Future object is a type of awaitable. Unlike a coroutine object when a future is awaited it does not cause a block of code to be executed. Instead a future object can be thought of as representing some process that is ongoing elsewhere and which may or may not yet be finished.

When you await a future the following happens:

  • If the process the future represents has finished and returned a value then the await statement immediately returns that value.
  • If the process the future represents has finished and raised an exception then the await statement immediately raises that exception.
  • If the process the future represents has not yet finished then the current Task is paused until the process has finished. Once it is finished it behaves as described in the first two bullet points here.

All Future objects f have the following synchronous interface in addition to being awaitable:

  • f.done() returns True if the process the future represents has finished.
  • f.exception() raises an asyncio.InvalidStateError exception if the process has not yet finished. If the process has finished it returns the exception it raised, or None if it terminated without raising.
  • f.result() raises an asyncio.InvalidStateError exception if the process has not yet finished. If the process has finished it raises the exception it raised, or returns the value it returned if it finished without raising.

It’s important to note that there is no way for a future that is done to ever change back into one that is not yet done. A future becoming done is a one-time occurrence.

IMPORTANT!: The distinction between a Coroutine and a Future is important. A Coroutine’s code will not be executed until it is awaited. A future represents something that is executing anyway, and simply allows your code to wait for it to finish, check if it has finished, and fetch the result if it has.
IMPORTANT!: Objects which implement the __await__ magic method may do almost anything when awaited. They might behave more like Coroutines, or more like Futures. They may do something else entirely. The documentation for the class in question should usually make it clear what their behaviour is.

You probably won’t create your own futures very often unless you are implementing new libraries that extend asyncio. However you will find that library functions often return futures. If you do need to create your own future directly you can do it with a call to

On the other hand you will probably find that you use a related method, create_task quite often …

TYPING NOTE: If you want to specify that a variable is a Future then you can use the asyncio.Future class as a type annotation. If you want to specify that the Future’s result should be of a specific type, R then you can use the following notation: f : asyncio . Future [ R ] (in Python 3.6 you will need to wrap asyncio.Future[R] in quotes for this to work correctly, but in later versions of Python this is no longer needed).

As described in the previous article each event loop contains a number of tasks, and every coroutine that is executing is doing so inside a task. So the question of how to create a task seems like an important one.

Creating a task is a simple matter, and can be done entirely in synchronous code:

NOTE: In Python 3.6 the function asyncio.create_task is not available, but you can still create a task using: t = asyncio . get_event_loop (). create_task ( example_coroutine_function ()) this is exactly the same, but a little more verbose.

The method create_task takes a coroutine object as a parameter and returns a Task object, which inherits from asyncio.Future . The call creates the task inside the event loop for the current thread, and starts the task executing at the beginning of the coroutine’s code-block. The returned future will be marked as done() only when the task has finished execution. As you might expect the return value of the coroutine’s code block is the result() which will be stored in the future object when it is finished (and if it raises then the exception will be caught and stored in the future).

Creating a task to wrap a coroutine is a synchronous call, so it can be done anywhere, including inside synchronous or asynchronous code. If you do it in asynchronous code then the event loop is already running (since it is currently executing your asynchronous code), and when it next gets the opportunity (ie. next time your current task pauses) it might make the new task active.

When you do it in synchronous code, however, chances are that the event loop is not yet running. Manualy manipulating event loops is discouranged by the python documentation. Unless you are developing libraries extending asyncio functionality, you should probably avoid trying to create a task from synchronous code.

If you do need to call a single piece of async code in an otherwise synchronous script, you can use asyncio.run() .

Running async programs

With the introduction of asyncio.run() in Python 3.7, and the removal of the loop parameter from many asyncio function in Python 3.10, managing event loops is something that you are unlikely to come across, unless you are developing an async library. The event loop objects are still there and accessible. There is a whole page in the docs discussing them. If you are working in Python 3.7 or greater, rejoice and give thanks for asyncio.run() .

asyncio.run(coro) will run coro , and return the result. It will always start a new event loop, and it cannot be called when the event loop is already running. This leads to a couple of obvious ways to run your async code.

The first is to have everything in async coroutines, and have a very simple entry function:

The second is to wrap each coroutine call in a separate run command. Note that this forgoes all of the benefits of asyncio. Still, there might be the odd script where this is the right thing to do.

Note that these simple examples don’t make use of the ability of async code to work on multiple tasks concurrently. A more sensible example is given at the end . As you work with asyncio in python, you’ll learn about more sophisticated ways to manage your work, but this is enough to get you started.

Manual event loop interaction

If you’re using Python 3.6, and you need to run coroutines from ordinary sync code (which you probably will, if you want to start something.) then you will need to start the event loop. There are two methods for doing this:

will cause the event loop to run forever (or until explicitly killed). This isn’t usually particularly useful. Much more useful is:

which takes a single parameter. If the parameter is a future (such as a task) then the loop will be run until the future is done, returning its result or raising its exception. So putting it together:

will create a new task which executes example_coroutine_function inside the event loop until it finishes, and then return the result.

In fact this can be simplified further since if you pass a coroutine object as the parameter to run_until_complete then it automatically calls create_task for you.

How to yield control

There is no simple command for yielding control to the event loop so that other tasks can run. In most cases in an asyncio program this is not something you will want to do explicitly, preferring to allow control to be yielded automatically when you await a future returned by some underlying library that handles some type of IO.

However occasionally you do need to, and in particular it’s quite useful during testing and debugging. As a result there is a recognised idiom for doing this if you need to. The statement:

will pause the current task and allow other tasks to be executed. The way this works is by using the function asyncio.sleep which is provided by the asyncio library. This function takes a single parameter which is a number of seconds, and returns a future which is not marked done yet but which will be when the specified number of seconds have passed.

Specifying a count of zero seconds works to interrupt the current task if other tasks are pending, but otherwise doesn’t do anything since the sleep time is zero.

The implementation of asyncio.sleep in the standard library has been optimised to make this an efficient operation.

When using asyncio.sleep with a non-zero parameter it’s worth noting that just because the future will become done when the number of seconds has passed does not mean that your task will always wake back up at that time. In fact it may wake back up at any point after that time, since it can only awaken when there’s no other task being run on the event loop.

  • You can only use the keywords await , async with and async for inside asynchronous code.
  • Asynchronous code must be contained inside an async def declaration (or one other place we’ll cover in the next article), but the declaration can go anywhere def is allowed.
  • The coroutine’s code will only be executed when it is awaited or wrapped in a task.
  • Awaiting a future will not cause code to be executed, but might pause your current task until another process has completed.
  • What happens then could be anything, check the documentation for the object in question.
  • You can wrap a coroutine in a task to make it execute and return a future you can use to monitor the results.

A diagram summarising the types of awaitable objects and their relationships. Coroutine inherits from Awaitable. Future also inherits from Awaitable. Task inherits from Future.

Making an actual program

So that concludes our run down of the basic syntax for writing asynchronous code. With just this you can already create a perfectly good async program which can instantiate multiple tasks and allow them to be swapped in and out. The following example is a fully working Python program using only the things included in this post:

This program will run four tasks which print the numbers from 0 to 99, and after printing each task will yield control to allow other tasks to take over. It neatly demonstrates that asyncio allows multiple things to be done interleaved.

To actually do anything useful you’ll need to make use of one of the libraries that implement io, such as aiohttp , and when you do you might well find that there are a few things in their interfaces which I haven’t covered in this post. Specifically you’ll probably find that the interface makes use of async with and possibly also async for . So those will be the subject of the next post in this series: Python Asyncio Part 3 – Asynchronous Context Managers and Asynchronous Iterators

  • Skip to main content
  • Skip to primary sidebar
  • Skip to footer

Additional menu

Super Fast Python Banner

Super Fast Python

making you awesome at concurrency

When Does Asyncio Switch Between Tasks

January 1, 2024 by Jason Brownlee in Python Asyncio

You may be wondering how asyncio chooses which task to run and how it switches between tasks.

This is an important question and highlights how asyncio tasks are different from typical Python functions and thread-based concurrency.

In this tutorial, you will discover how asyncio switches between tasks and coroutines and how this is different from the way we switch between threads.

Let’s get started.

Table of Contents

How Does Asyncio Switch Tasks?

A common question about asyncio that I’m asked is:

  • How does Python switch between tasks in asyncio?

Given that many developers have some knowledge of threading, they might ask:

  • Do asyncio tasks switch like Python threads.

This is a great question as it highlights the central difference between coroutines and threads.

To dig into this question we need to first review top-down forced context switching of threads in preemptive multitasking, and then contrast this to bottom-up volunteering in cooperating multitasking.

Run loops using all CPUs, download your FREE book to learn how.

Python Threads Context Switch

Python supports native threads that are managed by the underlying operating system.

The underlying operating system (e.g. Linux, MacOS, or Windows) manages which threads run at what time on the underlying CPU core hardware.

Typically, an operating system will be running many processes, each with one or more threads. There may be hundreds or thousands of threads running at a time as part of normal operation.

Not all threads are able to run at the same time. Instead, the operating system simulates multitasking by allowing each thread to run for a short amount of time before pausing the execution of the thread, storing its state, and switching to another thread.

The process of suspending one thread and reanimating a suspended thread is called a context switch .

In computing, a context switch is the process of storing the state of a process or thread, so that it can be restored and resume execution at a later point, and then restoring a different, previously saved, state. — Context switch, Wikipedia .

Python threads will be context-switched by the operating system at any time, although typically when blocked, such as reading or writing from a file or socket.

The Python interpreter also offers a context switch interval configuration which is a fixed number of Python bytecode instructions (e.g. 100) or time interval (e,g. 100 milliseconds) that may be executed before the interpreter will explicitly offer a thread up for a context switch.

This is called preemptive multitasking and is the way that the operating system is able to manage and execute a large number of threads concurrently.

Preemptive multitasking involves the use of an interrupt mechanism which suspends the currently executing process and invokes a scheduler to determine which process should execute next. Therefore, all processes will get some amount of CPU time at any given time. — Preemption (computing), Wikipedia .

You can learn more about context-switching threads and the context-switch interval in Python in the tutorial:

  • Context Switch Interval In Python

Download Now: Free Asyncio PDF Cheat Sheet

Asyncio Does Not Use Context Switching

Tasks and coroutines in asyncio are not context-switched by the operating system.

The asyncio event loop executes one task at a time.

A task will run for as long as it wants and then must explicitly yield control.

This will suspend the current task and allow the event loop to resume the execution of the next scheduled task.

This is called cooperative multitasking.

You can learn more about the explicit differences between asyncio and threading in the tutorial:

  • Asyncio vs Threading in Python

Free Python Asyncio Course

Download your FREE Asyncio PDF cheat sheet and get BONUS access to my free 7-day crash course on the Asyncio API.

Discover how to use the Python asyncio module including how to define, create, and run new coroutines and how to use non-blocking I/O.

Learn more  

What is Cooperative Multitasking

Cooperative multitasking is a concurrency model where multiple tasks or processes work together by voluntarily yielding control of the CPU to allow other tasks to execute.

Unlike preemptive multitasking, where an external scheduler forcibly switches between tasks based on a fixed time slice, cooperative multitasking relies on the cooperation of tasks to relinquish control.

Cooperative multitasking, also known as non-preemptive multitasking, is a style of computer multitasking in which the operating system never initiates a context switch from a running process to another process. Instead, in order to run multiple applications concurrently, processes voluntarily yield control periodically or when idle or logically blocked. — Cooperative multitasking, Wikipedia .

In cooperative multitasking, each task must explicitly yield control to the scheduler or other tasks when it reaches a certain point or completes its current work. This requires tasks to be well-behaved and cooperative, as a task that does not yield control can monopolize the CPU and hinder the execution of other tasks.

One of the key advantages of cooperative multitasking is its simplicity and low overhead.

Context switching between tasks is more lightweight compared to preemptive multitasking, as it doesn’t involve saving and restoring the complete CPU state. However, the main drawback is that a misbehaving or poorly designed task can potentially disrupt the entire system by not yielding control, leading to unresponsiveness.

Cooperative multitasking is commonly used with non-blocking socket I/O, where it’s used to manage asynchronous operations without the need for explicit multi-threading.

Developers need to ensure that their tasks cooperate and yield control appropriately to maintain smooth execution in cooperative multitasking systems.

Overwhelmed by the python concurrency APIs? Find relief, download my FREE Python Concurrency Mind Maps

How Can An Asyncio Task Yield Control

An asyncio task will yield control, or suspend when it awaits something.

Anytime a coroutine uses the “ await ” expression , such as on a task, it is signaling to the event loop that it is yielding control.

This might be for many reasons, such as:

  • Waiting for a result from a task or coroutine.
  • Waiting for a read or write operation on a socket or subprocess.
  • Waiting for an asynchronous generator.
  • Entering or exiting an asynchronous context manager.

A good place to start might be:

  • What is Async/Await in Python
  • What is Asyncio Await in Python

In fact, looking closely at any asyncio program, you will see a given coroutine is littered with explicit and implicit await expressions, points where the current task may suspend and the coroutine will resume the next scheduled task.

An asyncio task may also explicitly yield control in the middle of a task that does not have any reasonable await points.

This can be achieved by awaiting a call to asyncio.sleep() for zero seconds.

This is a no-operation (nop) that signals the event loop that the current task is happy to yield control if any other scheduled tasks need an opportunity to run.

You can learn more about asyncio.sleep(0) in the tutorial:

  • What is asyncio.sleep(0)

Python Asyncio Jump-Start

Loving The Tutorials?

Why not take the next step? Get the book.

What If Asyncio Tasks Don’t Yield Control

What if asyncio coroutines are selfish?

What if asyncio tasks never await anything?

This is a common concern by developers looking at asyncio through the lens of classical imperative or object-oriented programming. Asyncio is different, it uses an asynchronous programming model.

It is this difference in the framing of asyncio that causes the biggest misunderstandings of the module and in turn frustration when using it.

For example, see the tutorial:

  • Why Python Developers Hate Asyncio

Asyncio can fail to give up control. Coroutines can be selfish and uncooperative.

This can happen and it signals that an asyncio program has been written in a non-async manner. It does not use the tools of cooperating multitasking and may not be developed using the asynchronous programming model .

Asynchronous, in computer programming, refers to the occurrence of events independent of the main program flow and ways to deal with such events. These may be “outside” events such as the arrival of signals, or actions instigated by a program that take place concurrently with program execution, without the program blocking to wait for results. — Asynchrony (computer programming), Wikipedia .

In this case, the program may need to be updated to be more cooperative.

You can learn more about Python asynchronous programming in the tutorial:

  • Asynchronous Programming in Python

Perhaps the addition of asyncio.sleep(0) calls can be added.

Perhaps synchronous blocks of code can be executed in a thread that is external to the event loop and awaited on like an async task, using mechanisms like asyncio.to_thread() .

You can learn more about executing blocking tasks in external threads in the tutorial:

  • How to Use Asyncio to_thread()
  • How to Run Blocking Tasks in Asyncio

Further Reading

This section provides additional resources that you may find helpful.

Python Asyncio Books

  • Python Asyncio Mastery , Jason Brownlee ( my book! )
  • Python Asyncio Jump-Start , Jason Brownlee.
  • Python Asyncio Interview Questions , Jason Brownlee.
  • Asyncio Module API Cheat Sheet

I also recommend the following books:

  • Python Concurrency with asyncio , Matthew Fowler, 2022.
  • Using Asyncio in Python , Caleb Hattingh, 2020.
  • asyncio Recipes , Mohamed Mustapha Tahrioui, 2019.
  • Python Asyncio: The Complete Guide
  • Python Asynchronous Programming
  • asyncio — Asynchronous I/O
  • Asyncio Coroutines and Tasks
  • Asyncio Streams
  • Asyncio Subprocesses
  • Asyncio Queues
  • Asyncio Synchronization Primitives
  • Asynchronous I/O, Wikipedia .
  • Coroutine, Wikipedia .

You now know how asyncio switches between tasks and coroutines.

Did I make a mistake? See a typo? I’m a simple humble human. Correct me, please!

Do you have any additional tips? I’d love to hear about them!

Do you have any questions? Ask your questions in the comments below and I will do my best to answer.

Photo by Eurico Craveiro on Unsplash

Share this:

Related tutorials:.

' src=

About Jason Brownlee

Hi, my name is Jason Brownlee, Ph.D. and I’m the guy behind this website. I am obsessed with Python Concurrency.

I help python developers learn concurrency, super fast. Learn more .

Parallel Loops in Python

Discover how to run your loops in parallel, download your free book now:

Parallel Loops in Python

Your free book " Parallel Loops in Python " includes complete and working code templates that you can modify and use right now in your own projects.

Reader Interactions

Do you have any questions cancel reply, learn asyncio fast (without the frustration).

Python Asyncio Jump-Start

What if you could develop Python programs that were asynchronous from the start?

The asyncio module provides easy-to-use coroutine-based concurrency for asynchronous programming.

Introducing: " Python Asyncio Jump-Start ".

A new book designed to teach you the asyncio module step-by-step, super fast!

Insert/edit link

Enter the destination URL

Or link to existing content

Understanding Python's "yield" Keyword

python task yield

The yield keyword in Python is used to create generators. A generator is a type of collection that produces items on-the-fly and can only be iterated once. By using generators you can improve your application's performance and consume less memory as compared to normal collections, so it provides a nice boost in performance.

In this article we'll explain how to use the yield keyword in Python and what it does exactly. But first, let's study the difference between a simple list collection and generator, and then we will see how yield can be used to create more complex generators.

  • Differences Between a List and Generator

In the following script we will create both a list and a generator and will try to see where they differ. First we'll create a simple list and check its type:

When running this code you should see that the type displayed will be "list".

Now let's iterate over all the items in the squared_list .

The above script will produce following results:

Now let's create a generator and perform the same exact task:

To create a generator, you start exactly as you would with list comprehension, but instead you have to use parentheses instead of square brackets. The above script will display "generator" as the type for squared_gen variable. Now let's iterate over the generator using a for-loop.

The output will be:

The output is the same as that of the list. So what is the difference? One of the main differences lies in the way the list and generators store elements in the memory. Lists store all of the elements in memory at once, whereas generators "create" each item on-the-fly, displays it, and then moves to the next element, discarding the previous element from the memory.

One way to verify this is to check the length of both the list and generator that we just created. The len(squared_list) will return 5 while len(squared_gen) will throw an error that a generator has no length. Also, you can iterate over a list as many times as you want but you can iterate over a generator only once. To iterate again, you must create the generator again.

  • Using the Yield Keyword

Now we know the difference between simple collections and generators, let us see how yield can help us define a generator.

In the previous examples, we created a generator implicitly using the list comprehension style. However in more complex scenarios we can instead create functions that return a generator. The yield keyword, unlike the return statement, is used to turn a regular Python function in to a generator. This is used as an alternative to returning an entire list at once. This will be again explained with the help of some simple examples.

Again, let's first see what our function returns if we do not use the yield keyword. Execute the following script:

In this script a function cube_numbers is created that accepts a list of numbers, take their cubes and returns the entire list to the caller. When this function is called, a list of cubes is returned and stored in the cubes variable. You can see from the output that the returned data is in-fact a full list:

Now, instead of returning a list, let's modify the above script so that it returns a generator.

Check out our hands-on, practical guide to learning Git, with best-practices, industry-accepted standards, and included cheat sheet. Stop Googling Git commands and actually learn it!

In the above script, the cube_numbers function returns a generator instead of list of cubed number. It's very simple to create a generator using the yield keyword. Here we do not need the temporary cube_list variable to store cubed number, so even our cube_numbers method is simpler. Also, no return statement is needed, but instead the yield keyword is used to return the cubed number inside of the for-loop.

Now, when cube_number function is called, a generator is returned, which we can verify by running the code:

Even though we called the cube_numbers function, it doesn't actually execute at this point in time, and there are not yet any items stored in memory.

To get the function to execute, and therefore the next item from generator, we use the built-in next method. When you call the next iterator on the generator for the first time, the function is executed until the yield keyword is encountered. Once yield is found the value passed to it is returned to the calling function and the generator function is paused in its current state.

Here is how you get a value from your generator:

The above function will return "1". Now when you call next again on the generator, the cube_numbers function will resume executing from where it stopped previously at yield . The function will continue to execute until it finds yield again. The next function will keep returning cubed value one by one until all the values in the list are iterated.

Once all the values are iterated the next function throws a StopIteration exception. It is important to mention that the cubes generator doesn't store any of these items in memory, rather the cubed values are computed at runtime, returned, and forgotten. The only extra memory used is the state data for the generator itself, which is usually much less than a large list. This makes generators ideal for memory-intensive tasks.

Instead of always having to use the next iterator, you can instead use a "for" loop to iterate over a generators values. When using a "for" loop, behind the scenes the next iterator is called until all the items in the generator are iterated over.

  • Optimized Performance

As mentioned earlier, generators are very handy when it comes to memory-intensive tasks since they do not need to store all of the collection items in memory, rather they generate items on the fly and discards it as soon as the iterator moves to the next item.

In the previous examples the performance difference of a simple list and generator was not visible since the list sizes were so small. In this section we'll check out some examples where we can distinguish between the performance of lists and generators.

In the code below we will write a function that returns a list that contains 1 million dummy car objects. We will calculate the memory occupied by the process before and after calling the function (which creates the list).

Take a look at the following code:

Note : You may have to pip install psutil to get this code to work on your machine.

In the machine on which the code was run, following results were obtained (yours may look slightly different):

Before the list was created the process memory was 8 MB , and after the creation of list with 1 million items, the occupied memory jumped to 334 MB . Also, the time it took to create the list was 1.58 seconds.

Now, let's repeat the above process but replace the list with generator. Execute the following script:

Here we have to use the for car in car_list_gen(1000000) loop to ensure that all 1000000 cars are actually generated.

Following results were obtained by executing the above script:

From the output, you can see that by using generators the memory difference is much smaller than before (from 8 MB to 40 MB ) since the generators do not store the items in memory. Furthermore, the time taken to call the generator function was a bit faster as well at 1.37 seconds, which is about 14% faster than the list creation.

Hopefully from this article you have a better understanding of the yield keyword, including how it's used, what it's used for, and why you'd want to use it. Python generators are a great way to improve the performance of your programs and they're very simple to use, but understanding when to use them is the challenge for many novice programmers.

You might also like...

  • Hidden Features of Python
  • Python Docstrings
  • Handling Unix Signals in Python
  • The Best Machine Learning Libraries in Python
  • Guide to Sending HTTP Requests in Python with urllib3

Improve your dev skills!

Get tutorials, guides, and dev jobs in your inbox.

No spam ever. Unsubscribe at any time. Read our Privacy Policy.

In this article

python task yield

Building Your First Convolutional Neural Network With Keras

Most resources start with pristine datasets, start at importing and finish at validation. There's much more to know. Why was a class predicted? Where was...

David Landup

Data Visualization in Python with Matplotlib and Pandas

Data Visualization in Python with Matplotlib and Pandas is a course designed to take absolute beginners to Pandas and Matplotlib, with basic Python knowledge, and...

© 2013- 2024 Stack Abuse. All rights reserved.

Python Enhancement Proposals

  • Python »
  • PEP Index »

PEP 525 – Asynchronous Generators

Rationale and goals, asynchronous generators, support for asynchronous iteration protocol, finalization, asynchronous generator object, pyasyncgenasend and pyasyncgenathrow, new standard library functions and types, backwards compatibility, regular generators, improvements over asynchronous iterators, aiter() and anext() builtins, asynchronous list/dict/set comprehensions, asynchronous yield from, why the asend() and athrow() methods are necessary, implementation, acknowledgments.

PEP 492 introduced support for native coroutines and async / await syntax to Python 3.5. It is proposed here to extend Python’s asynchronous capabilities by adding support for asynchronous generators .

Regular generators (introduced in PEP 255 ) enabled an elegant way of writing complex data producers and have them behave like an iterator.

However, currently there is no equivalent concept for the asynchronous iteration protocol ( async for ). This makes writing asynchronous data producers unnecessarily complex, as one must define a class that implements __aiter__ and __anext__ to be able to use it in an async for statement.

Essentially, the goals and rationale for PEP 255 , applied to the asynchronous execution case, hold true for this proposal as well.

Performance is an additional point for this proposal: in our testing of the reference implementation, asynchronous generators are 2x faster than an equivalent implemented as an asynchronous iterator.

As an illustration of the code quality improvement, consider the following class that prints numbers with a given delay once iterated:

The same can be implemented as a much simpler asynchronous generator:

Specification

This proposal introduces the concept of asynchronous generators to Python.

This specification presumes knowledge of the implementation of generators and coroutines in Python ( PEP 342 , PEP 380 and PEP 492 ).

A Python generator is any function containing one or more yield expressions:

We propose to use the same approach to define asynchronous generators :

The result of calling an asynchronous generator function is an asynchronous generator object , which implements the asynchronous iteration protocol defined in PEP 492 .

It is a SyntaxError to have a non-empty return statement in an asynchronous generator.

The protocol requires two special methods to be implemented:

  • An __aiter__ method returning an asynchronous iterator .
  • An __anext__ method returning an awaitable object, which uses StopIteration exception to “yield” values, and StopAsyncIteration exception to signal the end of the iteration.

Asynchronous generators define both of these methods. Let’s manually iterate over a simple asynchronous generator:

PEP 492 requires an event loop or a scheduler to run coroutines. Because asynchronous generators are meant to be used from coroutines, they also require an event loop to run and finalize them.

Asynchronous generators can have try..finally blocks, as well as async with . It is important to provide a guarantee that, even when partially iterated, and then garbage collected, generators can be safely finalized. For example:

The above code defines an asynchronous generator that uses async with to iterate over a database cursor in a transaction. The generator is then iterated over with async for , which interrupts the iteration at some point.

The square_series() generator will then be garbage collected, and without a mechanism to asynchronously close the generator, Python interpreter would not be able to do anything.

To solve this problem we propose to do the following:

This is very similar to what the close() method does to regular Python generators, except that an event loop is required to execute aclose() .

  • Raise a RuntimeError , when an asynchronous generator executes a yield expression in its finally block (using await is fine, though): async def gen (): try : yield finally : await asyncio . sleep ( 1 ) # Can use 'await'. yield # Cannot use 'yield', # this line will trigger a # RuntimeError.
  • Add two new methods to the sys module: set_asyncgen_hooks() and get_asyncgen_hooks() .

The idea behind sys.set_asyncgen_hooks() is to allow event loops to intercept asynchronous generators iteration and finalization, so that the end user does not need to care about the finalization problem, and everything just works.

sys.set_asyncgen_hooks() accepts two arguments:

  • firstiter : a callable which will be called when an asynchronous generator is iterated for the first time.
  • finalizer : a callable which will be called when an asynchronous generator is about to be GCed.

When an asynchronous generator is iterated for the first time, it stores a reference to the current finalizer .

When an asynchronous generator is about to be garbage collected, it calls its cached finalizer . The assumption is that the finalizer will schedule an aclose() call with the loop that was active when the iteration started.

For instance, here is how asyncio is modified to allow safe finalization of asynchronous generators:

The second argument, firstiter , allows event loops to maintain a weak set of asynchronous generators instantiated under their control. This makes it possible to implement “shutdown” mechanisms to safely finalize all open generators and close the event loop.

sys.set_asyncgen_hooks() is thread-specific, so several event loops running in parallel threads can use it safely.

sys.get_asyncgen_hooks() returns a namedtuple-like structure with firstiter and finalizer fields.

The asyncio event loop will use sys.set_asyncgen_hooks() API to maintain a weak set of all scheduled asynchronous generators, and to schedule their aclose() coroutine methods when it is time for generators to be GCed.

To make sure that asyncio programs can finalize all scheduled asynchronous generators reliably, we propose to add a new event loop coroutine method loop.shutdown_asyncgens() . The method will schedule all currently open asynchronous generators to close with an aclose() call.

After calling the loop.shutdown_asyncgens() method, the event loop will issue a warning whenever a new asynchronous generator is iterated for the first time. The idea is that after requesting all asynchronous generators to be shutdown, the program should not execute code that iterates over new asynchronous generators.

An example of how shutdown_asyncgens coroutine should be used:

The object is modeled after the standard Python generator object. Essentially, the behaviour of asynchronous generators is designed to replicate the behaviour of synchronous generators, with the only difference in that the API is asynchronous.

The following methods and properties are defined:

  • agen.__aiter__() : Returns agen .
  • agen.__anext__() : Returns an awaitable , that performs one asynchronous generator iteration when awaited.
  • agen.aclose() : Returns an awaitable , that throws a GeneratorExit exception into the generator. The awaitable can either return a yielded value, if agen handled the exception, or agen will be closed and the exception will propagate back to the caller.
  • agen.__name__ and agen.__qualname__ : readable and writable name and qualified name attributes.
  • agen.ag_await : The object that agen is currently awaiting on, or None . This is similar to the currently available gi_yieldfrom for generators and cr_await for coroutines.
  • agen.ag_frame , agen.ag_running , and agen.ag_code : defined in the same way as similar attributes of standard generators.

StopIteration and StopAsyncIteration are not propagated out of asynchronous generators, and are replaced with a RuntimeError .

Implementation Details

Asynchronous generator object ( PyAsyncGenObject ) shares the struct layout with PyGenObject . In addition to that, the reference implementation introduces three new objects:

  • PyAsyncGenASend : the awaitable object that implements __anext__ and asend() methods.
  • PyAsyncGenAThrow : the awaitable object that implements athrow() and aclose() methods.
  • _PyAsyncGenWrappedValue : every directly yielded object from an asynchronous generator is implicitly boxed into this structure. This is how the generator implementation can separate objects that are yielded using regular iteration protocol from objects that are yielded using asynchronous iteration protocol.

PyAsyncGenASend and PyAsyncGenAThrow are awaitables (they have __await__ methods returning self ) and are coroutine-like objects (implementing __iter__ , __next__ , send() and throw() methods). Essentially, they control how asynchronous generators are iterated:

../_images/pep-0525-1.png

PyAsyncGenASend is a coroutine-like object that drives __anext__ and asend() methods and implements the asynchronous iteration protocol.

agen.asend(val) and agen.__anext__() return instances of PyAsyncGenASend (which hold references back to the parent agen object.)

The data flow is defined as follows:

Subsequent iterations over the PyAsyncGenASend objects, push None to agen .

When a _PyAsyncGenWrappedValue object is yielded, it is unboxed, and a StopIteration exception is raised with the unwrapped value as an argument.

  • return statements in asynchronous generators raise StopAsyncIteration exception, which is propagated through PyAsyncGenASend.send() and PyAsyncGenASend.throw() methods.

PyAsyncGenAThrow is very similar to PyAsyncGenASend . The only difference is that PyAsyncGenAThrow.send() , when called first time, throws an exception into the parent agen object (instead of pushing a value into it.)

  • types.AsyncGeneratorType – type of asynchronous generator object.
  • sys.set_asyncgen_hooks() and sys.get_asyncgen_hooks() methods to set up asynchronous generators finalizers and iteration interceptors in event loops.
  • inspect.isasyncgen() and inspect.isasyncgenfunction() introspection functions.
  • New method for asyncio event loop: loop.shutdown_asyncgens() .
  • New collections.abc.AsyncGenerator abstract base class.

The proposal is fully backwards compatible.

In Python 3.5 it is a SyntaxError to define an async def function with a yield expression inside, therefore it’s safe to introduce asynchronous generators in 3.6.

Performance

There is no performance degradation for regular generators. The following micro benchmark runs at the same speed on CPython with and without asynchronous generators:

The following micro-benchmark shows that asynchronous generators are about 2.3x faster than asynchronous iterators implemented in pure Python:

Design Considerations

Originally, PEP 492 defined __aiter__ as a method that should return an awaitable object, resulting in an asynchronous iterator.

However, in CPython 3.5.2, __aiter__ was redefined to return asynchronous iterators directly. To avoid breaking backwards compatibility, it was decided that Python 3.6 will support both ways: __aiter__ can still return an awaitable with a DeprecationWarning being issued.

Because of this dual nature of __aiter__ in Python 3.6, we cannot add a synchronous implementation of aiter() built-in. Therefore, it is proposed to wait until Python 3.7.

Syntax for asynchronous comprehensions is unrelated to the asynchronous generators machinery, and should be considered in a separate PEP.

While it is theoretically possible to implement yield from support for asynchronous generators, it would require a serious redesign of the generators implementation.

yield from is also less critical for asynchronous generators, since there is no need provide a mechanism of implementing another coroutines protocol on top of coroutines. And to compose asynchronous generators a simple async for loop can be used:

They make it possible to implement concepts similar to contextlib.contextmanager using asynchronous generators. For instance, with the proposed design, it is possible to implement the following pattern:

Another reason is that it is possible to push data and throw exceptions into asynchronous generators using the object returned from __anext__ object, but it is hard to do that correctly. Adding explicit asend() and athrow() will pave a safe way to accomplish that.

In terms of implementation, asend() is a slightly more generic version of __anext__ , and athrow() is very similar to aclose() . Therefore, having these methods defined for asynchronous generators does not add any extra complexity.

A working example with the current reference implementation (will print numbers from 0 to 9 with one second delay):

PEP 525 was accepted by Guido, September 6, 2016 [2] .

The implementation is tracked in issue 28003 [3] . The reference implementation git repository is available at [1] .

I thank Guido van Rossum, Victor Stinner, Elvis Pranskevichus, Nathaniel Smith, Łukasz Langa, Andrew Svetlov and many others for their feedback, code reviews, and discussions around this PEP.

This document has been placed in the public domain.

Source: https://github.com/python/peps/blob/main/peps/pep-0525.rst

Last modified: 2023-09-09 17:39:29 GMT

Python AsyncIO Awaitables: Coroutine, Future, and Task

Introduction

Python asyncio is a library for efficient single-thread concurrent applications. In my last blog post “Python AsyncIO Event Loop” , we have understood what an event loop is in Python asyncio by looking at the Python source code. This seems to be effective to understand how Python asyncio works.

In this blog post, I would like to take one step further and discuss the mechanisms of the three key asyncio awaitables, including Coroutine , Future , and Task , by looking at the Python source code again.

Starting from Python 3.5, coroutine functions are defined using async def and Coroutine objects are created by calling coroutine functions. The abstracted class of Coroutine is just as follows. It does not have method overloading because the derived class and method overload is generated by Python interpreter for the coroutine functions defined using async def . The key method for Coroutine class is send . It is trying to mimic the behavior of trampoline.

“Fortunately”, Python asyncio coroutine was once implemented using a @asyncio.coroutine decorator on a Python generator in Python 3.4. Hopefully the logic of the coroutine in Python 3.5+ is similar to the coroutine in Python 3.4 that it yields sub coroutine upon calling.

A typical coroutine could be implemented using a Python generator just like the follows.

The @asyncio.coroutine decorator implementation is as follows.

Without looking into the details, this @asyncio.coroutine decorator almost does not change the generator at all, since most likely wrapper $\approx$ coro .

When we tried to run coroutine with loop.run_until_complete , we see from the comment that if the argument is a coroutine then it would be converted to a Task in the first place, and loop.run_until_complete is actually scheduling Task s. So we would look into Task shortly.

Future has closed relationship with Task , so let’s look at Future first.

Future use has an event loop. By default, it is the event loop in the main thread.

The key method of Future is future.set_result . Let’s check what will happen if we call future.set_result .

Once future.set_result is called, it would trigger self.__schedule_callbacks asking the even loop to call all the callback s related to the Future as soon as possible. These Future related callback s are added or removed by future.add_done_callback or future.remove_done_callback . If no Future related callback s, no more callback s are scheduled in the event loop.

So we have known what will happen after the Future got result. What happens when the Future is scheduled in the event loop?

From the last blog post “Python AsyncIO Event Loop” , we have seen the Future was scheduled into the event loop via loop.ensure_future . “If the argument is a Future, it is returned directly.” So when the Future is scheduled in the event loop, there is almost no callback scheduled, until the future.set_result is called. (I said almost no callback because there is a default callback _run_until_complete_cb added as we have seen in the last blog post.)

Because _PyFuture = Future , Task is just a derived class of Future . The task of a Task is to wrap a coroutine in a Future .

In the constructor, we see that the Task schedules a callback self.__step in the event loop. The task.__step is a long method, but we should just pay attention to the try block and the else block since these two are the ones mostly likely to be executed.

Here we see the coroutine.send method again. Each time we call coroutine.send in the try block, we get a result . In the else blcok, we always have another self._loop.call_soon call. We do this in a trampoline fashion until Coroutine runs out of results to send .

Trampoline Function

The flavor of the wrapping of Task to Coroutine is somewhat similar to trampoline. Every time we call coroutine.send , we got some returned values and scheduled another callback .

The implementation of asyncio is complicated and I don’t expect I could know all the details. But trying to understand more about the low-level design might be useful for implementing low-level asyncio libraries and prevent stupid mistakes in high-level asyncio applications.

The key to scheduling the key asyncio awaitables, Coroutine , Future , and Task , are that the awaitables are all wrapped into Future in some way under the hood of asyncio interface.

https://leimao.github.io/blog/Python-AsyncIO-Awaitable-Coroutine-Future-Task/

Licensed under

Like this article support the author with.

  • 1 Introduction
  • 2 Coroutine
  • 5 Trampoline Function
  • 6 Conclusion

Advertisement

We Love Servers.

  • WHY IOFLOOD?
  • BARE METAL CLOUD
  • DEDICATED SERVERS

Python Yield | Keyword Guide (With Examples)

Artistic depiction of yielding values in Python featuring looping arrows and data generation markers emphasizing efficient data streaming

Ever wondered how to create a generator in Python? The ‘yield’ keyword, much like a magician’s wand, has the power to transform your functions into generators. Instead of producing a single value, your function now conjures up a sequence of results!

This guide is your roadmap to mastering the use of ‘yield’ in Python. We’ll start from the basics and gradually delve into more advanced techniques. By the end of this journey, you’ll be wielding ‘yield’ like a true Python wizard!

TL;DR: What does ‘yield’ do in Python?

‘Yield’ in Python is used to define a generator function, which can produce a sequence of results instead of a single value. Let’s see it in action with a simple example:

The function simple_generator is not just a regular function. It’s a generator function, thanks to the ‘yield’ keyword. When we loop over simple_generator() , it yields the numbers 1, 2, and 3, one at a time. This is just scratching the surface of Python’s ‘yield’. Continue reading for a deeper dive into more advanced usage scenarios and a comprehensive understanding of this powerful keyword.

Table of Contents

The Fundamentals of Python’s ‘yield’

Python yield: advanced techniques, exploring alternatives to python’s ‘yield’, navigating challenges with python’s ‘yield’, understanding generators and iteration in python, leveraging ‘yield’ in large-scale python projects, further reading: coroutines and the ‘itertools’ module, python yield: a powerful tool for generators.

In Python, ‘yield’ is the keyword that gives a function the power to become a generator. But what does that mean, exactly? Let’s break it down with a simple example.

In this code, simple_generator is a generator function. We use the ‘yield’ keyword to produce values one at a time. When we call next(gen) , it runs the function until it hits a ‘yield’ statement, then pauses and returns the yielded value. The next time we call next(gen) , it picks up where it left off and continues running until it hits the next ‘yield’ statement.

Advantages and Pitfalls of ‘yield’

One of the main advantages of using ‘yield’ is that it allows us to handle large data sets efficiently. When we use a generator function with ‘yield’, it doesn’t store all the values in memory at once. Instead, it generates them on the fly as we need them. This can be a huge advantage when working with large data sets.

However, there’s a potential pitfall to keep in mind. Once a generator function has yielded all of its values, it can’t be reused. If you try to get more values out of it, you’ll get a StopIteration exception. This is because a generator function maintains its state only until it has yielded all its values.

As we delve deeper into the world of Python’s ‘yield’, we encounter more complex and intriguing uses. Let’s explore two of these advanced techniques: creating infinite sequences and using ‘yield’ in recursive functions.

Infinite Sequences with ‘yield’

‘Yield’ can be used to create infinite sequences. This is possible because a generator function only generates the next value when it’s needed. Here’s an example of an infinite sequence generator:

In this code, the infinite_sequence generator function yields an infinite sequence of numbers. Each time we call next(gen) , it gives us the next number in the sequence.

Recursive Functions and ‘yield’

‘Yield’ can also be used in recursive functions to create more complex sequences. Here’s an example of a function that generates the Fibonacci sequence:

In this code, the fibonacci generator function uses ‘yield’ to produce the Fibonacci sequence. Each number is the sum of the two preceding ones, starting from 0 and 1.

These advanced techniques show the power and flexibility of Python’s ‘yield’. However, they also come with their own set of challenges. For instance, when creating infinite sequences, we must be careful to avoid entering an infinite loop. And when using ‘yield’ in recursive functions, we must ensure that the recursion has a base case to prevent it from running indefinitely.

While ‘yield’ is a powerful tool for creating generators in Python, it’s not the only method. Let’s explore two alternative approaches: generator expressions and built-in functions like ‘range’.

Generator Expressions: The Compact Alternative

Generator expressions are a high-performance, memory-efficient generalization of list comprehensions and lambdas. Here’s an example of a generator expression that produces the same sequence as our simple_generator function from earlier:

This generator expression does the same thing as the simple_generator function, but in a more compact form. However, generator expressions can be less flexible and harder to read when they get complex.

Built-in Functions: Python’s ‘range’

Python’s built-in range function is another way to create generators. range generates a sequence of numbers and is often used in loops. Here’s an example:

In this code, range(3) generates the numbers 0, 1, and 2. It’s a simple and efficient way to create a sequence of numbers, but it’s less flexible than ‘yield’ because it can only generate sequences of numbers.

In conclusion, while ‘yield’ is a powerful and flexible tool for creating generators in Python, generator expressions and built-in functions like ‘range’ offer alternative approaches. The best method to use depends on the specific needs of your code.

While ‘yield’ is a powerful tool for creating generators in Python, it’s not without its challenges. Let’s discuss some common issues and how to navigate them.

Understanding the ‘StopIteration’ Exception

One common issue when working with ‘yield’ is the StopIteration exception. This happens when you try to get a value from a generator that has already yielded all its values. Here’s an example:

In this code, the simple_generator function yields two values. When we try to get a third value with next(gen) , it raises a StopIteration exception because the generator has no more values to yield.

Handling ‘StopIteration’

One way to handle the StopIteration exception is to use a loop to iterate over the generator. The loop will automatically stop when the generator runs out of values. Here’s an example:

In this code, the for loop automatically stops when the simple_generator function runs out of values. This prevents the StopIteration exception from being raised.

Other Considerations

Another thing to keep in mind when using ‘yield’ is that generator functions maintain their state only until they have yielded all their values. Once a generator has been exhausted, it can’t be reused. If you need to use the same sequence of values again, you’ll need to create a new generator.

In conclusion, while ‘yield’ can be a powerful tool for creating generators in Python, it’s important to understand its potential pitfalls and how to navigate them.

Python’s ‘yield’ keyword is closely tied to the concept of generators and iteration. But what are these concepts, and how does ‘yield’ fit into them?

Generators: The Power Behind ‘yield’

In Python, a generator is a type of iterable, like a list or a tuple. But unlike lists or tuples, generators don’t store all their values in memory. Instead, they generate each value on-the-fly as you loop over them. This makes them much more memory-efficient when dealing with large sequences of data.

Here’s a simple example of a generator function:

In this code, the count_up_to function is a generator function. It uses the ‘yield’ keyword to produce a sequence of numbers from 1 up to n . Each time we loop over the generator, it yields the next number in the sequence.

Iteration: The Process that Drives ‘yield’

Iteration is the process of looping over an iterable. When you use a for loop to go over a list, you’re iterating over that list. When you use ‘yield’ in a function, you’re creating a generator that can be iterated over.

The power of ‘yield’ lies in its ability to pause the function’s execution after each ‘yield’ statement, and resume it the next time the generator’s next method is called. This allows the function to produce a sequence of values over time, rather than calculating them all at once and returning them in a huge list.

In conclusion, ‘yield’ plays a crucial role in creating generators and enabling iteration in Python. It provides a memory-efficient way to produce large or even infinite sequences of data, making it a powerful tool in any Python programmer’s toolkit.

The ‘yield’ keyword isn’t just for small scripts or simple sequences. It can also be a powerful tool in larger scripts or projects, especially when dealing with large data sets or creating data pipelines.

Data Pipelines with ‘yield’

In data analysis or machine learning projects, you often need to process large amounts of data. This data might need to be filtered, transformed, or otherwise processed before it can be used. ‘Yield’ can help you create efficient data pipelines for these tasks.

Here’s a simple example of a data pipeline that reads a large file line by line, filters out the empty lines, and yields the remaining lines one by one:

In this code, the read_large_file function is a generator that reads a large file line by line. It uses ‘yield’ to produce the lines one at a time, allowing you to process each line individually without loading the entire file into memory.

‘yield’ and Large Data Sets

When working with large data sets, memory efficiency is a key concern. ‘Yield’ allows you to create generators that produce data on-the-fly as you loop over them, making them much more memory-efficient than lists or other iterables that store all their values in memory.

If you’re interested in diving deeper into the world of generators and iteration in Python, two topics you might want to explore are coroutines and the ‘itertools’ module.

Coroutines are a more advanced form of generators that can not only produce values with ‘yield’, but also consume values sent to them using the .send() method. This makes them incredibly powerful for creating complex data pipelines or handling asynchronous tasks.

The ‘itertools’ module, on the other hand, provides a set of tools for creating and working with iterators. It includes functions for creating infinite iterators, cycling over iterables, and more.

Further Resources for Python Iterator Mastery

If you’re eager to expand your understanding of Python’s iterators, control statements and loops, look no further. Here are a few handpicked resources to stimulate your knowledge:

  • Python Loop Use Cases Unveiled – Dive into Python’s itertools module for advanced looping techniques.

Simplifying Loops with Python “while” Loop – Explore how “while” loops handle conditions and control program flow.

Python “pass” Statement: Do Nothing with Purpose – Discover the “pass” statement in Python and its use as a placeholder.

Codecademy’s Python Loops Cheatsheet is an easy-to-reference cheatsheet on loops in Python.

Python Generators Explanation – Codecademy’s Python documentation explaining the functionality of generators in Python.

Control Structures in Python – A JavaTpoint tutorial that provides a comprehensive overview of control structures in Python.

By delving into these resources and enhancing your understanding of Python’s control structure you can become a more adept Python developer.

Python’s ‘yield’ keyword is a powerful tool for creating generators. It allows a function to produce a sequence of results over time, rather than calculating them all at once and returning them in a list. This makes it incredibly memory-efficient, especially when dealing with large sequences of data.

From simple sequences to infinite and complex sequences, ‘yield’ offers a wide range of possibilities. It can be used in a simple function to create a generator, or in a recursive function to create more complex sequences. However, it also comes with challenges, such as understanding the StopIteration exception and ensuring the generator does not run indefinitely.

While ‘yield’ is a powerful tool, it’s not the only way to create generators in Python. Generator expressions offer a more compact alternative, while built-in functions like ‘range’ provide a simple and efficient way to create sequences of numbers.

Working with ‘yield’ can sometimes lead to issues like the StopIteration exception. This exception is raised when you try to get a value from a generator that has already yielded all its values. To handle this, you can use a loop to iterate over the generator, which will automatically stop when the generator runs out of values.

A Comparison of Generator Creation Methods:

In conclusion, ‘yield’ is a powerful and versatile tool in Python. Whether you’re creating a simple generator or a complex data pipeline, ‘yield’ can help you do it more efficiently and effectively.

About Author

Gabriel Ramuglia

Gabriel Ramuglia

Gabriel is the owner and founder of IOFLOOD.com , an unmanaged dedicated server hosting company operating since 2010.Gabriel loves all things servers, bandwidth, and computer programming and enjoys sharing his experience on these topics with readers of the IOFLOOD blog.

Related Posts

Boolean logic in Python true false operations check marks crosses Python code

Table of Contents

What is yield in python, generator functions in python, example of using yield in python (fibonacci series), how can you call functions using yield, why and when should you use yield, yield vs. return in python, advantages and disadvantages of yield, choose the right software development program, wrapping up, yield in python: an ultimate tutorial on yield keyword in python.

Yield in Python: An Ultimate Tutorial on Yield Keyword in Python

Python has tons of utilities that make the lives of developers exponentially easier. One such utility is the yield keyword in Python, which can be used to replace return statements that you use in normal functions in Python. This comprehensive article will explore everything about the yield keyword in Python and how it is used in generator functions. So with no further ado, let's get started.

The Yield keyword in Python is similar to a return statement used for returning values or objects in Python. However, there is a slight difference. The yield statement returns a generator object to the one who calls the function which contains yield, instead of simply returning a value. 

Inside a program, when you call a function that has a yield statement, as soon as a yield is encountered, the execution of the function stops and returns an object of the generator to the function caller. In simpler words, the yield keyword will convert an expression that is specified along with it to a generator object and return it to the caller. Hence, if you want to get the values stored inside the generator object, you need to iterate over it.

It will not destroy the local variables’ states. Whenever a function is called, the execution will start from the last yield expression. Please note that a function that contains a yield keyword is known as a generator function. 

When you use a function with a return value, every time you call the function, it starts with a new set of variables. In contrast, if you use a generator function instead of a normal function, the execution will start right from where it left last.

If you want to return multiple values from a function, you can use generator functions with yield keywords. The yield expressions return multiple values. They return one value, then wait, save the local state, and resume again. 

The general syntax of the yield keyword in Python is - 

Before you explore more regarding yield keywords, it's essential first to understand the basics of generator functions.

Want a Top Software Development Job? Start Here!

Want a Top Software Development Job? Start Here!

In Python, generator functions are those functions that, instead of returning a single value, return an iterable generator object. You can access or read the values returned from the generator function stored inside a generator object one-by-one using a simple loop or using next() or list() methods.

You can create a generator function using the generator() and yield keywords. Consider the example below.

def generator():

   yield "Welcome"

   yield "to"

   yield "Simplilearn"

gen_object = generator()

print(type(gen_object))

for i in gen_object:

   print(i)

In the above program, you have created a simple generator function and used multiple yield statements to return multiple values, which are stored inside a generator object when you create it. You can then loop over the object to print the values stored inside it.

YieldInPython.

Let’s create another generator function with yield keywords. You will try to filter out all the odd numbers from a list of numbers. Also, here it is essential to use different methods such as list(), for-in, and next() to output the values stored inside the generator object.

Consider the example below.

def filter_odd(numbers):

   for number in range(numbers):

       if(number%2!=0):

           yield number 

odd_numbers = filter_odd(20)

print(list(odd_numbers))

YieldInPython_2

You can see that it has printed the generator object as a list.

You can also use the for-in loop to print the values stored inside the generator object. Here is how to do so.

           yield number

for num in odd_numbers:

   print(num)

YieldInPython_3.

Finally, yet another method to print the elements stored inside a generator object is using the next() method. Each time you invoke the next() method on the generator object, it returns the next item.

print(next(odd_numbers))

YieldInPython_4

Please note that if there is no item left in the generator object and you invoke the next() method on it, it will return a StopIteration error.

Also, it’s very important to note that you can call the generators only once in the same program. Consider the program below.

for i in odd_numbers:

YieldInPython_5

You can see that first when you invoked the list method on the generator object, it returned the output. However, next time, when you used the for-in loop to print the values, it returned nothing. Hence, you can conclude that you can use the generator objects only once. If you want to use it again, you need to call it again.

Here is a general example that you can use to understand the concept of yield in the most precise manner. Here is a Fibonacci program that has been created using the yield keyword instead of return.

def fibonacci(n):

   temp1, temp2 = 0, 1

   total = 0

   while total < n:

       yield temp1

       temp3 = temp1 + temp2

       temp1 = temp2

       temp2 = temp3

       total += 1

fib_object = fibonacci(20)

print(list(fib_object))

Here, you have created a Fibonacci program that returns the top 20 Fibonacci numbers. Instead of storing each number in an array or list and then returning the list, you have used the yield method to store it in an object which saves a ton of memory, especially when the range is large.

YieldInPython_6

Instead of return values using yield, you can also call functions. For example, suppose you have a function called cubes which takes an input number and cubes it, and there exists another function that uses a yield statement to generate cubes of a range of numbers. In such a case, you can use the cubes function along with the yield statement to create a simple program. Let's check out the code below.

def cubes(number):

   return number*number*number

def getCubes(range_of_nums):

   for i in range(range_of_nums):

       yield cubes(i)

cube_object = getCubes(5)

print(list(cube_object))

YieldInPython_7

You can see how you can use yield to compute values by calling the function directly along with the statement and store them in a generator object.

When you use a yield keyword inside a generator function, it returns a generator object instead of values. In fact, it stores all the returned values inside this generator object in a local state. If you have used the return statement, which returned an array of values, this would have consumed a lot of memory. Hence, yield should always be preferred over the return in such cases. 

Moreover, the execution of the generator function starts only when the caller iterates over the generator object. Hence, it increases the overall efficiency of the program along with decreasing memory consumption. Some situations where you should use yield are - 

  • When the size of returned data is quite large, instead of storing them into a list, you can use yield.
  • If you want faster execution or computation over large datasets, yield is a better option.
  • If you want to reduce memory consumption, you can use yield.
  • It can be used to produce an infinite stream of data. You can set the size of a list to infinite, as it might cause a memory limit error.
  • If you want to make continuous calls to a function that contains a yield statement, it starts from the last defined yield statement, and hence, you can save a lot of time.

Before you understand the difference between yield and return in Python, it’s very important to understand the differences between a normal function that uses a return statement and a generator function that uses a yield statement.

A normal function directly stores and returns the value. However, generator functions return generator objects which contain all the values to be returned and they store them locally, thus reducing a lot of memory usage. 

Also, when you call a normal function, the execution stops as soon as it gets to the return statement. Hence, after starting, you can’t stop the execution of a normal function. However, in the case of generator functions, as soon as it reaches the first yield statement, it stops the execution and sends the value to the generator function. When the caller iterates over this value, then the next yield statement is processed, and the cycle continues. 

Below are some differences between yield and return in Python.

The advantages of using yield keywords instead of return are that the values returned by yield statement are stored as local variables states, which allows control over memory overhead allocation. Also, each time, the execution does not start from the beginning, since the previous state is retained.

However, a disadvantage of yield is that, if the calling of functions is not handled properly, the yield statements might sometimes cause errors in the program. Also, when you try to use the yield statements to improve time and space complexities, the overall complexity of the code increases which makes it difficult to understand.

This table compares various courses offered by Simplilearn, based on several key features and details. The table provides an overview of the courses' duration, skills you will learn, additional benefits, among other important factors, to help learners make an informed decision about which course best suits their needs.

Program Name Automation Testing Masters Program Full Stack Developer - MEAN Stack Caltech Coding Bootcamp Geo All All US University Simplilearn Simplilearn Caltech Course Duration 11 Months 11 Months 6 Months Coding Experience Required Basic Knowledge Basic Knowledge Basic Knowledge Skills You Will Learn Java, AWS, API Testing, TDD, etc. HTML, CSS, Express.js, API Testing, etc. Java, JavaScript, Angular, MongoDB, etc. Additional Benefits Structured Guidance Learn From Experts Hands-on Training Blended Learning Program Learn 20+ Tools and Skills Industry Aligned Projects Caltech Campus Connect Career Services 17 CEU Credits Cost $$ $$ $$$$ Explore Program Explore Program Explore Program

To sum up, you can leverage the yield statements in Python to return multiple values from generator functions. It is highly memory-efficient and increases the overall performance of the code. It saves memory because it stores the values to be returned as local variables state, and also each time it executes the function, it need not start from the beginning as the previous states are retained. This is what makes yield keywords highly popular among developers and a great alternative to return statements.

In this tutorial, you explored how you can leverage yield in Python to optimize programs in terms of both speed and memory. You saw several examples of generator functions and the different scenarios where you can use the yield statements. Moreover, you also explored why and when should you use it, along with its advantages and disadvantages.

You differentiated the use of normal functions and generator functions, and at the same time, you also compared return statements with yield keywords.

We hope that this comprehensive tutorial will give you better in-depth insights into yield keywords in Python.

If you are looking to learn further and master python and get started on your journey to becoming a Python expert, Simplilearn's Caltech Coding Bootcamp should be your next step. This comprehensive course gives you the work-ready training you need to master python including key topics like data operations , shell scripting, and conditional statement. You even get a practical hands-on exposure to Djang in this course.

If on the other hand, you have any queries or feedback for us on this yield in python article, do mention them in the comments section at the end of this page. We will review them and respond to you at the earliest. 

Happy Learning!

Our Software Development Courses Duration And Fees

Software Development Course typically range from a few weeks to several months, with fees varying based on program and institution.

Recommended Reads

Python Interview Guide

An Introduction to Enumerate in Python with Syntax and Examples

The Best Guide for RPA Using Python

Azure Functions: A Comprehensive Guide for Beginners

What Is Python typeof Function?

The Complete Guide to Data Visualization in Python

Get Affiliated Certifications with Live Class programs

Caltech coding bootcamp, automation testing masters program.

  • Comprehensive blended learning program
  • 200 hours of Applied Learning

Post Graduate Program in Full Stack Web Development

  • Live sessions on the latest AI trends, such as generative AI, prompt engineering, explainable AI, and more
  • Caltech CTME Post Graduate Certificate
  • PMP, PMI, PMBOK, CAPM, PgMP, PfMP, ACP, PBA, RMP, SP, and OPM3 are registered marks of the Project Management Institute, Inc.

Python Tutorial

File handling, python modules, python numpy, python pandas, python matplotlib, python scipy, machine learning, python mysql, python mongodb, python reference, module reference, python how to, python examples, python yield keyword.

❮ Python Keywords

Return three values from a function:

Definition and Usage

The yield keyword is used to return a list of values from a function.

Unlike the return keyword which stops further execution of the function, the yield keyword continues to the end of the function.

When you call a function with yield keyword(s), the return value will be a list of values, one for each yield .

Related Pages

Use the return keyword to return only one value, and stop further execution.

Read more about functions in our Python Functions Tutorial .

Get Certified

COLOR PICKER

colorpicker

Report Error

If you want to report an error, or if you want to make a suggestion, do not hesitate to send us an e-mail:

[email protected]

Top Tutorials

Top references, top examples, get certified.

PythonFixing

Saturday, November 20, 2021

[fixed] python: task got bad yield:.

  November 20, 2021       python-3.x , python-asyncio , twisted       No comments    

I am aiming to make parallel requests to the list of endpoints, hence using asyncio ensure_future can somone please take a look and give me an idea on how to fix errors (python3.6.7)

The problem is that you should not use asyncio with treq .

According to documentation:

treq depends on a recent Twisted and functions on Python 2.7 and Python 3.3+ (including PyPy).

If you want to use asyncio you have to use some other http client framework e.g. aiohttp.

If you need any example on how to use aiohttp client, feel free to ask.

  • Share This:  
  •  Facebook
  •  Twitter
  •  Google+
  •  Stumble

0 comments:

Post a comment.

Note: Only a member of this blog may post a comment.

Popular Posts

  • [FIXED] Flask- The requested URL was not found on the server Issue I think this is a simple problem but I am kind of stuck here. I am trying to deploy...
  • [FIXED] Selenium driver.Url vs. driver.Navigate().GoToUrl() Issue Which is the preferred method to open a Url (and are there any differences behind th...
  • [FIXED] Can't write in csv file Issue When I try to write the information in the csv file, error is thrown: Traceback (mo...

python task yield

  • [FIXED] ERROR: Could not build wheels for pycairo, which is required to install pyproject.toml-based projects Issue Error while installing manimce, I have been trying to install manimce library on win...

python task yield

  • [FIXED] Failed to start the Kernel - Jupyter in VS Code Issue I am trying to use a Jupyter notebook for some Pandas in VS Code. I set up a virtual...
  • [FIXED] How to enable/disable easily shared axis using matplotlib Issue The previous answer from tcaswell on how to create shared axis for plots which are ...

python task yield

The Shift from Models to Compound AI Systems

AI caught everyone’s attention in 2023 with Large Language Models (LLMs) that can be instructed to perform general tasks, such as translation or coding, just by prompting. This naturally led to an intense focus on models as the primary ingredient in AI application development, with everyone wondering what capabilities new LLMs will bring. As more developers begin to build using LLMs, however, we believe that this focus is rapidly changing: state-of-the-art AI results are increasingly obtained by compound systems with multiple components, not just monolithic models .

For example, Google’s AlphaCode 2 set state-of-the-art results in programming through a carefully engineered system that uses LLMs to generate up to 1 million possible solutions for a task and then filter down the set. AlphaGeometry , likewise, combines an LLM with a traditional symbolic solver to tackle olympiad problems. In enterprises, our colleagues at Databricks found that 60% of LLM applications use some form of retrieval-augmented generation (RAG) , and 30% use multi-step chains. Even researchers working on traditional language model tasks, who used to report results from a single LLM call, are now reporting results from increasingly complex inference strategies: Microsoft wrote about a chaining strategy that exceeded GPT-4’s accuracy on medical exams by 9%, and Google’s Gemini launch post measured its MMLU benchmark results using a new CoT@32 inference strategy that calls the model 32 times, which raised questions about its comparison to just a single call to GPT-4. This shift to compound systems opens many interesting design questions, but it is also exciting, because it means leading AI results can be achieved through clever engineering, not just scaling up training.

In this post, we analyze the trend toward compound AI systems and what it means for AI developers. Why are developers building compound systems? Is this paradigm here to stay as models improve? And what are the emerging tools for developing and optimizing such systems—an area that has received far less research than model training? We argue that compound AI systems will likely be the best way to maximize AI results in the future , and might be one of the most impactful trends in AI in 2024.

python task yield

Why Use Compound AI Systems?

We define a Compound AI System as a system that tackles AI tasks using multiple interacting components, including multiple calls to models, retrievers, or external tools. In contrast, an AI Model is simply a statistical model , e.g., a Transformer that predicts the next token in text.

Even though AI models are continually getting better, and there is no clear end in sight to their scaling, more and more state-of-the-art results are obtained using compound systems. Why is that? We have seen several distinct reasons:

  • Some tasks are easier to improve via system design. While LLMs appear to follow remarkable scaling laws that predictably yield better results with more compute, in many applications, scaling offers lower returns-vs-cost than building a compound system. For example, suppose that the current best LLM can solve coding contest problems 30% of the time, and tripling its training budget would increase this to 35%; this is still not reliable enough to win a coding contest! In contrast, engineering a system that samples from the model multiple times, tests each sample, etc. might increase performance to 80% with today’s models, as shown in work like AlphaCode . Even more importantly, iterating on a system design is often much faster than waiting for training runs. We believe that in any high-value application, developers will want to use every tool available to maximize AI quality, so they will use system ideas in addition to scaling. We frequently see this with LLM users, where a good LLM creates a compelling but frustratingly unreliable first demo, and engineering teams then go on to systematically raise quality.
  • Systems can be dynamic. Machine learning models are inherently limited because they are trained on static datasets, so their “knowledge” is fixed. Therefore, developers need to combine models with other components, such as search and retrieval, to incorporate timely data. In addition, training lets a model “see” the whole training set, so more complex systems are needed to build AI applications with access controls (e.g., answer a user’s questions based only on files the user has access to).
  • Improving control and trust is easier with systems. Neural network models alone are hard to control: while training will influence them, it is nearly impossible to guarantee that a model will avoid certain behaviors. Using an AI system instead of a model can help developers control behavior more tightly, e.g., by filtering model outputs. Likewise, even the best LLMs still hallucinate, but a system combining, say, LLMs with retrieval can increase user trust by providing citations or automatically verifying facts .
  • Performance goals vary widely. Each AI model has a fixed quality level and cost, but applications often need to vary these parameters. In some applications, such as inline code suggestions, the best AI models are too expensive, so tools like Github Copilot use carefully tuned smaller models and various search heuristics to provide results. In other applications, even the largest models, like GPT-4, are too cheap! Many users would be willing to pay a few dollars for a correct legal opinion, instead of the few cents it takes to ask GPT-4, but a developer would need to design an AI system to utilize this larger budget.

The shift to compound systems in Generative AI also matches the industry trends in other AI fields, such as self-driving cars: most of the state-of-the-art implementations are systems with multiple specialized components ( more discussion here ). For these reasons, we believe compound AI systems will remain a leading paradigm even as models improve.

Developing Compound AI Systems

While compound AI systems can offer clear benefits, the art of designing, optimizing, and operating them is still emerging. On the surface, an AI system is a combination of traditional software and AI models, but there are many interesting design questions. For example, should the overall “control logic” be written in traditional code (e.g., Python code that calls an LLM), or should it be driven by an AI model (e.g. LLM agents that call external tools)? Likewise, in a compound system, where should a developer invest resources—for example, in a RAG pipeline, is it better to spend more FLOPS on the retriever or the LLM, or even to call an LLM multiple times? Finally, how can we optimize an AI system with discrete components end-to-end to maximize a metric, the same way we can train a neural network? In this section, we detail a few example AI systems, then discuss these challenges and recent research on them.

The AI System Design Space

Below are few recent compound AI systems to show the breadth of design choices:

Key Challenges in Compound AI Systems

Compound AI systems pose new challenges in design, optimization and operation compared to AI models.

Design Space

The range of possible system designs for a given task is vast. For example, even in the simple case of retrieval-augmented generation (RAG) with a retriever and language model, there are: (i) many retrieval and language models to choose from, (ii) other techniques to improve retrieval quality, such as query expansion or reranking models, and (iii) techniques to improve the LLM’s generated output (e.g., running another LLM to check that the output relates to the retrieved passages). Developers have to explore this vast space to find a good design.

In addition, developers need to allocate limited resources, like latency and cost budgets, among the system components. For example, if you want to answer RAG questions in 100 milliseconds, should you budget to spend 20 ms on the retriever and 80 on the LLM, or the other way around?

Optimization

Often in ML, maximizing the quality of a compound system requires co-optimizing the components to work well together. For example, consider a simple RAG application where an LLM sees a user question, generates a search query to send to a retriever, and then generates an answer. Ideally, the LLM would be tuned to generate queries that work well for that particular retriever , and the retriever would be tuned to prefer answers that work well for that LLM .

In single model development a la PyTorch, users can easily optimize a model end-to-end because the whole model is differentiable. However, compound AI systems contain non-differentiable components like search engines or code interpreters, and thus require new methods of optimization. Optimizing these compound AI systems is still a new research area; for example, DSPy offers a general optimizer for pipelines of pretrained LLMs and other components, while others systems, like LaMDA , Toolformer and AlphaGeometry , use tool calls during model training to optimize models for those tools.

Machine learning operations (MLOps) become more challenging for compound AI systems. For example, while it is easy to track success rates for a traditional ML model like a spam classifier, how should developers track and debug the performance of an LLM agent for the same task, which might use a variable number of “reflection” steps or external API calls to classify a message? We believe that a new generation of MLOps tools will be developed to tackle these problems. Interesting problems include:

  • Monitoring: How can developers most efficiently log, analyze, and debug traces from complex AI systems?
  • DataOps: Because many AI systems involve data serving components like vector DBs, and their behavior depends on the quality of data served, any focus on operations for these systems should additionally span data pipelines.
  • Security: Research has shown that compound AI systems, such as an LLM chatbot with a content filter, can create unforeseen security risks compared to individual models. New tools will be required to secure these systems.

Emerging Paradigms

To tackle the challenges of building compound AI systems, multiple new approaches are arising in the industry and in research. We highlight a few of the most widely used ones and examples from our research on tackling these challenges.

Designing AI Systems: Composition Frameworks and Strategies. Many developers are now using “language model programming” frameworks that let them build applications out of multiple calls to AI models and other components. These include component libraries like LangChain and LlamaIndex that developers call from traditional programs, agent frameworks like AutoGPT and BabyAGI that let an LLM drive the application, and tools for controlling LM outputs, like Guardrails , Outlines , LMQL and SGLang . In parallel, researchers are developing numerous new inference strategies to generate better outputs using calls to models and tools, such as chain-of-thought , self-consistency , WikiChat , RAG and others.

Automatically Optimizing Quality: DSPy. Coming from academia, DSPy is the first framework that aims to optimize a system composed of LLM calls and other tools to maximize a target metric. Users write an application out of calls to LLMs and other tools, and provide a target metric such as accuracy on a validation set, and then DSPy automatically tunes the pipeline by creating prompt instructions, few-shot examples, and other parameter choices for each module to maximize end-to-end performance. The effect is similar to end-to-end optimization of a multi-layer neural network in PyTorch , except that the modules in DSPy are not always differentiable layers. To do that, DSPy leverages the linguistic abilities of LLMs in a clean way: to specify each module, users write a natural language signature, such as user_question -> search_query , where the names of the input and output fields are meaningful, and DSPy automatically turns this into suitable prompts with instructions, few-shot examples, or even weight updates to the underlying language models.

Optimizing Cost: FrugalGPT and AI Gateways. The wide range of AI models and services available makes it challenging to pick the right one for an application. Moreover, different models may perform better on different inputs. FrugalGPT is a framework to automatically route inputs to different AI model cascades to maximize quality subject to a target budget. Based on a small set of examples, it learns a routing strategy that can outperform the best LLM services by up to 4% at the same cost, or reduce cost by up to 90% while matching their quality. FrugalGPT is an example of a broader emerging concept of AI gateways or routers, implemented in software like Databricks AI Gateway , OpenRouter , and Martian , to optimize the performance of each component of an AI application. These systems work even better when an AI task is broken into smaller modular steps in a compound system, and the gateway can optimize routing separately for each step.

Operation: LLMOps and DataOps. AI applications have always required careful monitoring of both model outputs and data pipelines to run reliably. With compound AI systems, however, the behavior of the system on each input can be considerably more complex, so it is important to track all the steps taken by the application and intermediate outputs. Software like LangSmith , Phoenix Traces , and Databricks Inference Tables can track, visualize and evaluate these outputs at a fine granularity, in some cases also correlating them with data pipeline quality and downstream metrics. In the research world, DSPy Assertions seeks to leverage feedback from monitoring checks directly in AI systems to improve outputs, and AI-based quality evaluation methods like MT-Bench , FAVA and ARES aim to automate quality monitoring.

Generative AI has excited every developer by unlocking a wide range of capabilities through natural language prompting. As developers aim to move beyond demos and maximize the quality of their AI applications, however, they are increasingly turning to compound AI systems as a natural way to control and enhance the capabilities of LLMs. Figuring out the best practices for developing compound AI systems is still an open question, but there are already exciting approaches to aid with design, end-to-end optimization, and operation. We believe that compound AI systems will remain the best way to maximize the quality and reliability of AI applications going forward, and may be one of the most important trends in AI in 2024.

BibTex for this post:

IMAGES

  1. Yield in Python—Make Your Functions Efficient

    python task yield

  2. Introduction to yield in Python with example

    python task yield

  3. How to use the yeild statement in Python?

    python task yield

  4. Python yield

    python task yield

  5. How to use the yeild statement in Python?

    python task yield

  6. How To Use yield in Python

    python task yield

VIDEO

  1. Python Overrated #python #pythonprogramming #pythontutorial

  2. Sales Prediction Using Python

  3. What Does The Yield Keyword Do In Python

  4. "yield" & "next" function in Python Programming 🌟#yield #next #function #python #youtubeshorts

  5. A Real Python Optimization You Can Use

  6. ⏳ Python Task 19: Countdown to Happiness! 🚀🎉 #python #task

COMMENTS

  1. Coroutines and Tasks

    async def main(): task1 = asyncio.create_task( say_after(1, 'hello')) task2 = asyncio.create_task( say_after(2, 'world')) print(f"started at {time.strftime('%X')}") # Wait until both tasks are completed (should take # around 2 seconds.) await task1 await task2 print(f"finished at {time.strftime('%X')}")

  2. What does the "yield" keyword do in Python?

    What does the "yield" keyword do in Python? Ask Question Asked 15 years, 4 months ago Modified 26 days ago Viewed 3.3m times 12864 Want to improve this post? Provide detailed answers to this question, including citations and an explanation of why your answer is correct. Answers without enough detail may be edited or deleted.

  3. Async IO in Python: A Complete Walkthrough

    Async IO is a concurrent programming design that has received dedicated support in Python, evolving rapidly from Python 3.4 through 3.7, and probably beyond. You may be thinking with dread, "Concurrency, parallelism, threading, multiprocessing. That's a lot to grasp already. Where does async IO fit in?"

  4. Getting Started With Async Features in Python

    Remove ads Have you heard of asynchronous programming in Python? Are you curious to know more about Python async features and how you can use them in your work? Perhaps you've even tried to write threaded programs and run into some issues. If you're looking to understand how to use Python async features, then you've come to the right place.

  5. How to Use Generators and yield in Python

    Example 1: Reading Large Files Example 2: Generating an Infinite Sequence Example 3: Detecting Palindromes Understanding Generators Building Generators With Generator Expressions Profiling Generator Performance Understanding the Python Yield Statement Using Advanced Generator Methods How to Use .send () How to Use .throw () How to Use .close ()

  6. Python asyncio.create_task(): Run Multiple Tasks Concurrently

    Notice that the task is different from the await keyword that blocks the entire coroutine until the operation completes with a result. It's important that you can create multiple tasks and schedule them to run instantly on the event loop at the same time. To create a task, you pass a coroutine to the create_task () function of the asyncio ...

  7. Python: Using the "yield" keyword with async/await (3 examples)

    In Python, the yield keyword is used to create a generator, which is a special kind of iterator that can be paused and resumed. In Python 3.6 and above, you can use the yield keyword inside an async function to create an asynchronous generator, which is an iterator that can also perform asynchronous operations.

  8. Python Generators/Coroutines/Async IO with examples

    1. Introduction Following is a brief overview on the terminologies used on the topics. Concurrency vs Parallelism Concurrency — ability to execute two or more tasks which can start, run and...

  9. Python Asyncio Part 2

    Creating a task is a simple matter, and can be done entirely in synchronous code: async def example_coroutine_function(): ... t = asyncio.create_task(example_coroutine_function()) NOTE: In Python 3.6 the function asyncio.create_task is not available, but you can still create a task using:

  10. When Does Asyncio Switch Between Tasks

    An asyncio task will yield control, or suspend when it awaits something. Anytime a coroutine uses the " await " expression, such as on a task, it is signaling to the event loop that it is yielding control. Waiting for a result from a task or coroutine. Waiting for a read or write operation on a socket or subprocess.

  11. Understanding Python's "yield" Keyword

    The yield keyword, unlike the return statement, is used to turn a regular Python function in to a generator. This is used as an alternative to returning an entire list at once. This will be again explained with the help of some simple examples. Again, let's first see what our function returns if we do not use the yield keyword.

  12. PEP 525

    A Python generator is any function containing one or more yield expressions: def func(): # a function return def genfunc(): # a generator function yield We propose to use the same approach to define asynchronous generators:

  13. Python AsyncIO Awaitables: Coroutine, Future, and Task

    The flavor of the wrapping of Task to Coroutine is somewhat similar to trampoline. Every time we call coroutine.send, we got some returned values and scheduled another callback.. Conclusion. The implementation of asyncio is complicated and I don't expect I could know all the details. But trying to understand more about the low-level design might be useful for implementing low-level asyncio ...

  14. Python Yield

    By delving into these resources and enhancing your understanding of Python's control structure you can become a more adept Python developer. Python Yield: A Powerful Tool for Generators. Python's 'yield' keyword is a powerful tool for creating generators.

  15. Yield in Python: An Ultimate Tutorial on Yield Keyword in Python

    The yield expressions return multiple values. They return one value, then wait, save the local state, and resume again. The general syntax of the yield keyword in Python is -. >>> yield expression. Before you explore more regarding yield keywords, it's essential first to understand the basics of generator functions.

  16. Python yield Keyword

    x = myFunc () for z in x: print(z) Try it Yourself » Definition and Usage The yield keyword is used to return a list of values from a function. Unlike the return keyword which stops further execution of the function, the yield keyword continues to the end of the function.

  17. Python Exercises, Practice, Challenges

    These free exercises are nothing but Python assignments for the practice where you need to solve different programs and challenges. All exercises are tested on Python 3. Each exercise has 10-20 Questions. The solution is provided for every question. Practice each Exercise in Online Code Editor. These Python programming exercises are suitable ...

  18. [FIXED] python: Task got bad yield: ~ PythonFixing

    cc by-sa 3.0. .doc .htaccess .ico .net .net-core .net-interactive 2-satisfiability 2captcha 2d 32bit-64bit 3d 3d-convolution 3gp 4d 7zip 960.gs a-star aar abc abort abseil absl-py absolute-value abstract-base-class abstract-class abstract-methods abstract-syntax-tree abstractuser accelerate accelerate-framework accelerometer accent-sensitive ...

  19. How does a threading.Thread yield the rest of its quantum in Python?

    python multithreading yield Share Improve this question Follow asked Apr 24, 2009 at 22:29 Tom Future 1,920 2 15 15 Add a comment 4 Answers Sorted by: 92 time.sleep (0) is sufficient to yield control -- no need to use a positive epsilon. Indeed, time.sleep (0) MEANS "yield to whatever other thread may be ready". Share Improve this answer Follow

  20. The Shift from Models to Compound AI Systems

    state-of-the-art AI results are increasingly obtained by compound systems with multiple components, not just monolithic models. For example, Google's AlphaCode 2 set state-of-the-art results in programming through a carefully engineered system that uses LLMs to generate up to 1 million possible solutions for a task and then filter down the set. ...

  21. python: Task got bad yield:

    1 Answer Sorted by: 1 The problem is that you should not use asyncio with treq. According to documentation: treq depends on a recent Twisted and functions on Python 2.7 and Python 3.3+ (including PyPy). If you want to use asyncio you have to use some other http client framework e.g. aiohttp.