Goal We are going to solve the inter-temporal choice problem, for ten time stamps, and perform some numerical optimization of the results Main Methods We do this by solving backwards. We will create a variable k to measure asset, and k_{t} the remaining asset at time t. Let us first declare the function for power utility. k is our asset holding, \gamma our relative margin of risk, and U the power utility. The power utility function is defined by:

Implementing Power Utility # risk aversion y = var("y", latex_name="\gamma", domain='real') # discount factor d = var("d", latex_name="\delta", domain='real') # final value at time t=f k_f = var("k_f", latex_name="k_f", domain='real') # the instrument's percentage return over a period: i.e. (1+mu)*I_t = k_{t+1} m = var("m", latex_name="\mu", domain='real') # boundary conditions assume(y>0) assume(y<1) assume(d>0) assume(d<1) # power utility u(c) = ((c^(1-y)-1)/(1-y)) u c |--> -(c^(-y + 1) - 1)/(y - 1) End Boundary Conditions At the final time stamp, we desire to consume all of our assets. Therefore, we will seed our investment amount at I=0. We will optimize for eventual global utility, therefore, we will talley our utility; starting this talley at 0.

at the final time, leave nothing for investment I=0; u_total = 0 Bottom-Up Dynamic Programming From every step from here, we will discount this utility by d, then solve for the previous step’s target consumption that would maximize utility. That is, at every step, we desire:

“we want to consume all we have that needed’t to be invested” and \max u(c_{t}) Recall also that (1+\mu)I_{t} = k_{t+1} (as \mu is the mean log-return, 1+that times I, how much was invested at time t, is the expected capital one time period from then.) Therefore, to make sure that k_{f} gets back in the final period, we solve for our seed value for I, how much to invest would be:

Actual implementation # create an dictionary to keep track of all the capital variables k = {} # we will iterate time stamps 1-10 T = 10 # a variable for captial at that time for i in range(T): k_t = var(f"k_{T-i}", latex_name=f"k_{T-i}") # t-i becasue we are solving backwards; i0 = T10 # what can be consumed at every time stamp # is the k of the previous timestamp, minus # what needs to be left over # we multiply here by d because we want to # discount future utility u_total = du_total + u(k_t-I) # add the current variable to dictionary k[T-i] = k_t # recall again i0=T10 because backwards # solve for the next investment amount I = k_t/(1+m) u_total -(((((((d(d*(k_10^(-y + 1) - 1)/(y - 1) + ((k_9 - k_10/(m + 1))^(-y + 1) - 1)/(y - 1)) + ((k_8 - k_9/(m + 1))^(-y + 1) - 1)/(y - 1))*d + ((k_7 - k_8/(m + 1))^(-y + 1) - 1)/(y - 1))*d + ((k_6 - k_7/(m + 1))^(-y + 1) - 1)/(y - 1))*d + ((k_5 - k_6/(m + 1))^(-y + 1) - 1)/(y - 1))*d + ((k_4 - k_5/(m + 1))^(-y + 1) - 1)/(y - 1))*d + ((k_3 - k_4/(m + 1))^(-y + 1) - 1)/(y - 1))*d + ((k_2 - k_3/(m + 1))^(-y + 1) - 1)/(y - 1))*d - ((k_1 - k_2/(m + 1))^(-y + 1) - 1)/(y - 1) Optimization with some constants We can now use the scipy numerical optimizer to minimize this target. Recall that we can recover the actual value of consumption at each step as c=k-\frac{k}{m+1}. We will set some initial conditions: _m = 0.01 # 1% period-to-period increase _k = 1000 # $1000 capital _y = 0.8 # generally risk averse _d = 0.9 # the future matters slightly less Optimization Target Function Recall that the scipy optimizer MINIMIZES, so we will make the loss the negative of utility. Before we finally start, we need to make the actual, numerical loss function that performs the substitution. The code is actually just doing some function substitution, so its not very exciting.

we reverse the k_* variables because it is stored in the dictionary # in reverse, because we knew the reverse condition first optim_variables = list(k.values()) optim_variables.reverse() # this function is also the callback, so it returning # True terminates execution def u_total_loss(x): # the optimizer's current step # we want to take [1:], because we need to keep k1 the same at _k the # initial value substitution_dict = {key: val for key, val in zip(optim_variables[1:], x)} # initial conditions substitution_dict[m] = _m substitution_dict[y] = _y substitution_dict[d] = _d substitution_dict[d] = _d # we want to keep the initial value k1 the same substitution_dict[k[1]] = _k try: # get value content = (-1*u_total).subs(substitution_dict) # recall we multiply by -1 because we are MINIMIZING, so the loss is # the inverse of the maximization utility target return float(content.n()), False except: return 0, True Optimize! Finally, we are ready to start. We will now create the other initial conditions k1…k10 and : we will set the initial value to all be 1000 (i.e. do nothing) and have the optimizer work it out from there:

from scipy.optimize import minimize target = minimize(lambda x:u_total_loss(x)[0], [_k for _ in range(T-1)], callback=lambda x:u_total_loss(x)[1]) target fun: -50.71592850322347 hess_inv: array(9518.97596212, 7617.14636381, 5964.42171873, 4433.87331935, 4253.91810669, 3528.72923763, 2329.61846616, 1769.85078017, 1126.51562458], ... [ 1126.51562458, 1359.86586059, 1639.19265606, 2255.16306648, 2598.27394651, 2293.45506703, 2657.8129831 , 2422.66762041, 2911.30717272) jac: array([ 0.00000000e+00, -3.81469727e-06, 2.38418579e-06, 0.00000000e+00, 8.58306885e-06, -5.24520874e-06, 2.86102295e-06, -1.43051147e-06, -5.24520874e-06]) message: 'Optimization terminated successfully.' nfev: 1360 nit: 130 njev: 136 status: 0 success: True x: array([841.22097906, 699.82556541, 573.89912346, 461.63474591, 361.51493714, 272.10309839, 192.29084196, 120.94057011, 57.12129925]) Recovering Actual Dollar Consumption Amount Awesome! We now can recover c at each point by a nice helpful function: c(k0, k1) = k0 - k1/(_m+1) “Consumption is how much we have, minus how much we would be investing” So, let us translate our list to the actual values consumed: capital_over_time = [_k]+target.x.tolist() # we need to add the initial condition _k back to the # inventory list consumption_over_time = [c(i,j) for i,j in zip(capital_over_time, capital_over_time[1:])] consumption_over_time [167.107941529699, 148.324379643989, 131.608611479784, 116.835018601197, 103.699164584812, 92.1059288329209, 81.7161261514775, 72.5477032414004, 64.3848282779261] Examples of Output The next set of slides show examples of possible optimization outputs—how decisions by the inter-temporal choice problem changes based on the inputs. Risk Averse _m = 0.01 # 1% period-to-period increase _k = 1000 # 1000 capital _y = 0.8 # generally risk averse _d = 0.9 # the future matters slightly less [167.107941529699, 148.324379643989, 131.608611479784, 116.835018601197, 103.699164584812, 92.1059288329209, 81.7161261514775, 72.5477032414004, 64.3848282779261] More Return Winning Game _m = 0.1 # 1% period-to-period increase _k = 1000 #1000 capital _y = 0.8 # generally risk averse _d = 0.9 # the future matters slightly less [154.860597149863, 152.989432556196, 151.010433069881, 149.201249715528, 147.329750167852, 145.539019666462, 143.739371599600, 141.984228587213, 140.243839963791] More Risk _m = 0.01 # 1% period-to-period increase _k = 1000 # 1000 capital _y = 0.2 # generally risky _d = 0.9 # the future matters slightly less [388.525041338376, 241.124420093987, 149.632568775223, 92.8644259086613, 57.6330459746870, 35.7667230511026, 22.1970017374152, 13.7754327365677, 8.54930907023498] Loosing Game _m = -0.01 # this is a loosing stock _k = 1000 #1000 capital _y = 0.9 # very safe _d = 0.9 # the future matters fun: 0 hess_inv: array(1, 0, 0, 0, 0, 0, 0, 0, 0], [0, 1, 0, 0, 0, 0, 0, 0, 0], [0, 0, 1, 0, 0, 0, 0, 0, 0], [0, 0, 0, 1, 0, 0, 0, 0, 0], [0, 0, 0, 0, 1, 0, 0, 0, 0], [0, 0, 0, 0, 0, 1, 0, 0, 0], [0, 0, 0, 0, 0, 0, 1, 0, 0], [0, 0, 0, 0, 0, 0, 0, 1, 0], [0, 0, 0, 0, 0, 0, 0, 0, 1) jac: array([0., 0., 0., 0., 0., 0., 0., 0., 0.]) message: 'Optimization terminated successfully.' nfev: 10 nit: 0 njev: 1 status: 0 success: True x: array([1000., 1000., 1000., 1000., 1000., 1000., 1000., 1000., 1000.]) Evidently: do nothing if we have a loosing cause. Winning Game _m = 1.00 # this is SUPER winning stock _k = 1000 # $1000 capital _y = 0.9 # very safe _d = 0.9 # the future matters [125.667556437602, 241.474827418105, 460.068836905327, 868.972817783791, 4540.45893314523, 4219.93058738029, 3988.05775624984, 3996.89431939885, 3615.74982832315] We made so much money that we are spending a lot of it and still spending it.