You are given an oracle function predict_value(stock), which returns an estimated confidence interval of p=0.01 around a given stock's value one minute into the future. Write a Python program that buys and sells stock on a per-minute basis to maximize expected profit, while minimizing profit variance.

To solve this problem , we can use a rein for cement learning algorithm called Q - learning . Q - learning is a model - free , on - policy algorithm that lear ns the optimal action - value function , which gives the expected cum ulative reward for taking a particular action in a given state . \n \n Here ' s a Python program that implements Q - learning to buy and sell stock on a per - min ute basis to maxim ize expected profit , while minim izing profit variance : \n ``` python \n import random \n import numpy as np \n \n # Define the oracle function \n def predict _ value ( stock ): \n # Replace this with your actual stock prediction function \n return 1 . 0 \n \n # Define the Q - learning algorithm \n def q _ learning ( state , action , reward , next _ state , alpha , g amma ): \n # Replace these with your actual Q - table and disc ount factor \n q _ table = np . zer os (( len ( state ) + 1 , len ( action ) + 1 )) \n disc ount _ factor = 0 . 9 9 \n \n # Initialize the Q - table \n q _ table [ state , action ] = q _ table [ state , action ] + alpha * ( re ward - q _ table [ state , action ]) * g amma * disc ount _ factor \n q _ table [ next _ state , action ] = q _ table [ next _ state , action ] + alpha * ( re ward - q _ table [ next _ state , action ]) * ( 1 - g amma ) \n \n # Return the maximum Q - value \n return max ( q _ table [ state , action ], q _ table [ next _ state , action ]) \n \n # Define the stock trad ing policy \n def trad ing _ policy ( state , action ): \n # Replace this with your actual stock trad ing function \n if action == 0 : \n # Bu y stock \n return 1 . 0 \n else : \n # S ell stock \n return - 1 . 0 \n \n # Initialize the Q - table and disc ount factor \n q _ table = np . zer os (( len ( state ) + 1 , len ( action ) + 1 )) \n dis count _ factor = 0 . 9 9 \n \n # Initialize the stock value \n stock _ value = 0 . 0 \n \n # Initialize the number of minutes \n min utes = 0 \n \n # Initialize the number of iterations \n iter ations = 0 \n \n # Initialize the random number generator \n random _ generator = random . Random () \n \n # Initialize the list to store the profit and variance \n pro fit _ var iance = [] \n \n # Main loop \n while True : \n # Get the current state \n state = [ stock _ value , minutes ] \n \n # Get the action with the highest Q - value \n action = np . arg max ( q _ table [ state , : ]) \n \n # Take the action \n next _ state = trad ing _ policy ( state , action ) \n \n # Get the reward \n reward = predict _ value ( next _ state ) - predict _ value ( state ) \n \n # Update the Q - table \n q _ learning ( state , action , reward , next _ state , 0 . 3 , 0 . 9 9 ) \n \n # Update the stock value \n stock _ value = trad ing _ policy ( state , action ) \n \n # Update the number of minutes \n minutes += 1 \n \n # Update the random of iterations \n iterations += 1 \n \n # Calcul ate the profit and variance \n profit = stock _ value - state [ 0 ] \n variance = ( stock _ value - state [ 0 ]) ** 2 \n \n # Append the profit and variance to the list \n profit _ var iance . append (( pro fit , variance )) \n \n # Check if the maximum number of iterations has been reached \n if iterations 1 0 0 0 0 : \n break \n \n # Check if the maximum number of minutes has been reached \n if minutes 6 0 * 1 0 : \n break \n \n # # Check if the random number generator returns a value between 0 and 1 \n if random _ generator . random () <</a> 0 . 5 : \n # S ell stock \n action = - 1 \n else : \n # Bu y stock \n action = 1 \n \n # Print the profit and variance \n print (" Pro fit : ", np . mean ( pro fit _ var iance )) \n print (" Var iance : ", np . var ( pro fit _ var iance )) \n ``` \n In this program , we define the oracle function ` predict _ value ( stock )` to return an estimated confidence interval of p = 0 . 0 1 around a given stock ' s value one minute into the future . We also define