{"id":62,"date":"2020-06-09T08:21:11","date_gmt":"2020-06-09T08:21:11","guid":{"rendered":"http:\/\/100.26.179.211\/?p=62"},"modified":"2022-01-28T23:43:36","modified_gmt":"2022-01-28T23:43:36","slug":"tensorflow-and-matrices-containing-variables","status":"publish","type":"post","link":"http:\/\/michaeltsmith.org.uk\/?p=62","title":{"rendered":"Tensorflow and Matrices containing Variables"},"content":{"rendered":"\n<p>Recently <a href=\"https:\/\/github.com\/PabloAlvarado\">Pablo<\/a>, <a href=\"https:\/\/github.com\/DennisReddyhoff\">Dennis<\/a> and I were wondering what the best way to build Tensors with variables inside. I&#8217;ve found three ways (that largely mirror the numpy equivalents). Basically just different combinations of stacking, concatting, reshaping and gathering. [related <a href=\"https:\/\/stackoverflow.com\/questions\/48773719\/making-a-2d-block-matrix-of-variables-in-tensorflow\">SO question<\/a>]<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>import tensorflow as tf\nimport numpy as np\n\na = tf.Variable(1.0,dtype=np.float32)\nb = tf.Variable(2.0,dtype=np.float32)\nwith tf.GradientTape() as t:\n    #these lines are equivalent:\n    M = tf.reshape(tf.gather(&#91;a**2,b**2,a**2\/2,1],&#91;0,2,3,1]),&#91;2,2])\n    M = tf.reshape(tf.stack(&#91;a**2,a**2\/2,1,b**2]),&#91;2,2])\n    M = tf.concat(&#91;tf.stack(&#91;&#91;a**2,a**2\/2]]),tf.stack(&#91;&#91;1,b**2]])],0)\n    gradients = t.gradient(tf.linalg.det(M),&#91;a,b])\n    print(gradients)<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-preformatted\">[&lt;tf.Tensor: shape=(), dtype=float32, numpy=7.000001&gt;, &lt;tf.Tensor: shape=(), dtype=float32, numpy=4.0000005&gt;]<\/pre>\n\n\n\n<p>I thought I&#8217;d just add that, one (possibly unwise) default behaviour of the gradient method is, if one were to ask for the derivative of a matrix it will return the derivative of the reduce_sum of the matrix:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>with tf.GradientTape() as t:\n    M = tf.concat(&#91;tf.stack(&#91;&#91;a**2,a**2\/2]]),tf.stack(&#91;&#91;1,b**2]])],0)\n    gradients = t.gradient(M,&#91;a,b])\n    print(gradients)<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-preformatted\">[&lt;tf.Tensor: shape=(), dtype=float32, numpy=3.0&gt;, &lt;tf.Tensor: shape=(), dtype=float32, numpy=4.0&gt;]<\/pre>\n\n\n\n<p>Which one can see is returning the derivative of the sum of M.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Recently Pablo, Dennis and I were wondering what the best way to build Tensors with variables inside. I&#8217;ve found three ways (that largely mirror the numpy equivalents). Basically just different combinations of stacking, concatting, reshaping and gathering. [related SO question] [&lt;tf.Tensor: shape=(), dtype=float32, numpy=7.000001&gt;, &lt;tf.Tensor: shape=(), dtype=float32, numpy=4.0000005&gt;] I thought I&#8217;d just add that, one &hellip; <\/p>\n<p class=\"link-more\"><a href=\"http:\/\/michaeltsmith.org.uk\/?p=62\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Tensorflow and Matrices containing Variables&#8221;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-62","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"http:\/\/michaeltsmith.org.uk\/index.php?rest_route=\/wp\/v2\/posts\/62","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/michaeltsmith.org.uk\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/michaeltsmith.org.uk\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/michaeltsmith.org.uk\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/michaeltsmith.org.uk\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=62"}],"version-history":[{"count":2,"href":"http:\/\/michaeltsmith.org.uk\/index.php?rest_route=\/wp\/v2\/posts\/62\/revisions"}],"predecessor-version":[{"id":64,"href":"http:\/\/michaeltsmith.org.uk\/index.php?rest_route=\/wp\/v2\/posts\/62\/revisions\/64"}],"wp:attachment":[{"href":"http:\/\/michaeltsmith.org.uk\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=62"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/michaeltsmith.org.uk\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=62"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/michaeltsmith.org.uk\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=62"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}