Lecture 12

Requesting New Actions
Current action set
New action
Mandel, Liu, Brunskil & Popovic, AAAI 2017
Expected Local Improvement
Prob. human gives you action ah for state s
π = f ( θi )
Teach a learner with policy π in environment for T steps, observe reward R
Create new training point [f(θi),R]
Reward Signal?
● Balance post test performance with amount of practice needed ● ps=Performance on skill s, ● p = Post test performance across all skills, ● l = # practices for skill s
Humans are Invention Machines
New actions
New sensors
Invention Machines: Creating Systems that Can Evolve Beyond Their Original Capacity To Reach Extraordinary Performance
Mostly Bad Human Input
Mandel, Liu, Brunskil & Popovic, AAAI 2017
• New actions = new hints • Learning where to ask for new hints

Another type of structural cell wall protein, called extensin, can form covalent bonds with other extensin proteins through the amino acid tyrosine. In extensin, the tyrosines are evenly spaced and when they bond with tyrosine on another extensin molecule, they can wrap around other cell wall constituents "knitting" the wall together.
Cell walls also contain functional proteins. Enzymatic activities in cell walls include: •Oxidative enzymes - peroxidases•Hydrolytic enzymes - pectinases, cellulases•"Expansins" - enzymes that catalyze cell wall "creep" activity General functions of cell wall enzymes include:
galacturonic acid
Pectic acid with salt bridges
4. Pectin - polymer of around 200 galacturonic acid molecules - many of the carboxyl groups are methylated (COOCH3) - less hydrated then pectic acid but soluble in hot water - another major component of middle lamella but also found in primary walls

Lecture 12: Visualizing and UnderstandingAdministrativeMilestones due tonight on Canvas, 11:59pmMidterm grades released on Gradescope this weekA3 due next Friday, 5/26HyperQuest deadline extended to Sunday 5/21, 11:59pm Poster session is June 6Last Time: Lots of Computer Vision TasksClassification + LocalizationSemantic SegmentationObject DetectionInstance SegmentationCATGRASS , CAT , TREE , SKYDOG , DOG , CATDOG , DOG , CATSingle ObjectMultiple ObjectNo objects, just pixelsThis image is CC0 public domainThis image is CC0 public domainWhat’s going on inside ConvNets?This image is CC0 public domainClass Scores:1000 numbers Input Image:3 x 224 x 224What are the intermediate features looking for?Krizhevsky et al, “ImageNet Classification with Deep Convolutional Neural Networks”, NIPS 2012.Figure reproduced with permission.First Layer: Visualize FiltersAlexNet:64 x 3 x 11 x 11ResNet-18:64 x 3 x 7 x 7ResNet-101:64 x 3 x 7 x 7DenseNet-121:64 x 3 x 7 x 7Krizhevsky, “One weird trick for parallelizing convolutional neural networks”, arXiv 2014 He et al, “Deep Residual Learning for Image Recognition”, CVPR 2016Huang et al, “Densely Connected Convolutional Networks”, CVPR 2017Visualize the filters/kernels (raw weights)We can visualize filters at higher layers, but not that interesting (these are taken from ConvNetJS CIFAR-10 demo)layer 1 weightslayer 2 weightslayer 3 weights16 x 3 x 7 x 720 x 16 x 7 x 720 x 20 x 7 x 7FC7 layerLast Layer4096-dimensional feature vector for an image (layer immediately before the classifier)Run the network on many images, collect the feature vectorsLast Layer: Nearest Neighbors4096-dim vectorTest image L2 Nearest neighbors in feature spaceRecall: Nearest neighborsin pixel spaceKrizhevsky et al, “ImageNet Classification with Deep Convolutional Neural Networks”, NIPS 2012.Figures reproduced with permission.Visualize the “space” of FC7 feature vectors by reducing dimensionality of vectors from 4096 to 2 dimensionsSimple algorithm: Principle Component Analysis (PCA) More complex: t-SNEVan der Maaten and Hinton, “Visualizing Data using t-SNE”, JMLR 2008Figure copyright Laurens van der Maaten and Geoff Hinton, 2008. Reproduced with permission.Van der Maaten and Hinton, “Visualizing Data using t-SNE”, JMLR 2008Krizhevsky et al, “ImageNet Classification with Deep Convolutional Neural Networks”, NIPS 2012. Figure reproduced with permission.See high-resolution versions at/people/karpathy/cnnembed/Visualizing ActivationsYosinski et al, “Understanding Neural Networks Through Deep Visualization”, ICML DL Workshop 2014.Figure copyright Jason Yosinski, 2014. Reproduced with permission.conv5 feature map is128x13x13; visualizeas 128 13x13grayscale imagesMaximally Activating PatchesPick a layer and a channel; e.g. conv5 is128 x 13 x 13, pick channel 17/128Run many images through the network,record values of chosen channelVisualize image patches that correspondto maximal activationsSpringenberg et al, “Striving for Simplicity: The All Convolutional Net”, ICLR Workshop 2015Figure copyright Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, Martin Riedmiller, 2015;reproduced with permission.Occlusion Experiments Mask part of the image beforefeeding to CNN, draw heatmap ofprobability at each mask locationZeiler and Fergus, “Visualizing and Understanding Convolutional Networks”, ECCV 2014Boat image is CC0 public domain Elephant image is CC0 public domain Go-Karts image is CC0 public domainHow to tell which pixels matter for classification?Dog Simonyan, Vedaldi, and Zisserman, “Deep Inside Convolutional Networks: Visualising Image Classification Modelsand Saliency Maps”, ICLR Workshop 2014.Figures copyright Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman, 2014; reproduced with permission.How to tell which pixels matter for classification?Dog Compute gradient of (unnormalized) classscore with respect to image pixels, takeabsolute value and max over RGB channelsSimonyan, Vedaldi, and Zisserman, “Deep Inside Convolutional Networks: Visualising Image Classification Modelsand Saliency Maps”, ICLR Workshop 2014.Figures copyright Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman, 2014; reproduced with permission.Simonyan, Vedaldi, and Zisserman, “Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps”, ICLR Workshop 2014.Figures copyright Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman, 2014; reproduced with permission.Saliency Maps: Segmentation without supervision Simonyan, Vedaldi, and Zisserman, “Deep Inside Convolutional Networks: Visualising Image Classification Modelsand Saliency Maps”, ICLR Workshop 2014.Figures copyright Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman, 2014; reproduced with permission.Rother et al, “Grabcut: Interactive foreground extraction using iterated graph cuts”, ACM TOG 2004Use GrabCut onsaliency mapPick a single intermediate neuron, e.g. onevalue in 128 x 13 x 13 conv5 feature mapCompute gradient of neuron value with respectto image pixelsZeiler and Fergus, “Visualizing and Understanding Convolutional Networks”, ECCV 2014Springenberg et al, “Striving for Simplicity: The All Convolutional Net”, ICLR Workshop 2015Pick a single intermediate neuron, e.g. onevalue in 128 x 13 x 13 conv5 feature mapCompute gradient of neuron value with respectto image pixelsImages come out nicer if you onlybackprop positive gradients througheach ReLU (guided backprop)ReLUZeiler and Fergus, “Visualizing and Understanding Convolutional Networks”, ECCV 2014 Springenberg et al, “Striving for Simplicity: The All Convolutional Net”, ICLR Workshop 2015Figure copyright Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, Martin Riedmiller, 2015; reproduced with permission.Zeiler and Fergus, “Visualizing and Understanding Convolutional Networks”, ECCV 2014Springenberg et al, “Striving for Simplicity: The All Convolutional Net”, ICLR Workshop 2015Figure copyright Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, Martin Riedmiller, 2015; reproduced with permission.(Guided) backprop: Find the part of an image that a neuron responds to Gradient ascent: Generate a synthetic image that maximally activates a neuronI* = arg maxIf(I) + R(I)Neuron value Natural image regularizer1.Initialize image to zerosscore for class c (before Softmax) zero imageRepeat:2. Forward image to compute current scores3. Backprop to get gradient of neuron value with respect to image pixels4. Make a small update to the imageSimple regularizer: Penalize L2norm of generated imageSimonyan, Vedaldi, and Zisserman, “Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps”, ICLR Workshop 2014.Figures copyright Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman, 2014; reproduced with permission.Simple regularizer: Penalize L2norm of generated imageSimonyan, Vedaldi, and Zisserman, “Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps”, ICLR Workshop 2014.Figures copyright Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman, 2014; reproduced with permission.Simple regularizer: Penalize L2norm of generated imageYosinski et al, “Understanding Neural Networks Through Deep Visualization”, ICML DL Workshop 2014. Figure copyright Jason Yosinski, Jeff Clune, Anh Nguyen, Thomas Fuchs, and Hod Lipson, 2014. Reproduced with permission.Better regularizer: Penalize L2 norm of image; also during optimization periodically(1)Gaussian blur image(2)Clip pixels with small values to 0(3)Clip pixels with small gradients to 0 Yosinski et al, “Understanding Neural Networks Through Deep Visualization”, ICML DL Workshop 2014.Better regularizer: Penalize L2 norm ofimage; also during optimizationperiodically(1)Gaussian blur image(2)Clip pixels with small values to 0(3)Clip pixels with small gradients to 0Yosinski et al, “Understanding Neural Networks Through Deep Visualization”, ICML DL Workshop 2014.Figure copyright Jason Yosinski, Jeff Clune, Anh Nguyen, Thomas Fuchs, and Hod Lipson, 2014. Reproduced with permission.Better regularizer: Penalize L2 norm ofimage; also during optimizationperiodically(1)Gaussian blur image(2)Clip pixels with small values to 0(3)Clip pixels with small gradients to 0Yosinski et al, “Understanding Neural Networks Through Deep Visualization”, ICML DL Workshop 2014.Figure copyright Jason Yosinski, Jeff Clune, Anh Nguyen, Thomas Fuchs, and Hod Lipson, 2014. Reproduced with permission.Use the same approach to visualize intermediate featuresYosinski et al, “Understanding Neural Networks Through Deep Visualization”, ICML DL Workshop 2014.Figure copyright Jason Yosinski, Jeff Clune, Anh Nguyen, Thomas Fuchs, and Hod Lipson, 2014. Reproduced with permission.Use the same approach to visualize intermediate featuresYosinski et al, “Understanding Neural Networks Through Deep Visualization”, ICML DL Workshop 2014.Figure copyright Jason Yosinski, Jeff Clune, Anh Nguyen, Thomas Fuchs, and Hod Lipson, 2014. Reproduced with permission.Adding “multi-faceted” visualization gives even nicer results:(Plus more careful regularization, center-bias)Nguyen et al, “Multifaceted Feature Visualization: Uncovering the Different Types of Features Learned By Each Neuron in Deep Neural Networks”, ICML Visualization for Deep Learning Workshop 2016. Figures copyright Anh Nguyen, Jason Yosinski, and Jeff Clune, 2016; reproduced with permission.Figures copyright Anh Nguyen, Jason Yosinski, and Jeff Clune, 2016; reproduced with permission.Optimize in FC6 latent space instead of pixel space:Nguyen et al, “Synthesizing the preferred inputs for neurons in neural networks via deep generator networks,” NIPS 2016Figure copyright Nguyen et al, 2016; reproduced with permission.(1)Start from an arbitrary image(2)Pick an arbitrary class(3)Modify the image to maximize the class(4)Repeat until network is fooledBoat image is CC0 public domain Elephant image is CC0 public domainWhat is going on? Ian Goodfellow will explain Boat image is CC0 public domainElephant image is CC0 public domainRather than synthesizing an image to maximize a specific neuron, insteadtry to amplify the neuron activations at some layer in the networkChoose an image and a layer in a CNN; repeat:1.Forward: compute activations at chosen layer2.Set gradient of chosen layer equal to its activation3.Backward: Compute gradient on image4.Update imageMordvintsev, Olah, and Tyka, “Inceptionism: Going Deeper into NeuralNetworks”, Google Research Blog. Images are licensed under CC-BY4.0Rather than synthesizing an image to maximize a specific neuron, instead try to amplify the neuron activations at some layer in the networkEquivalent to:I* = arg max I ∑i f i (I)2Mordvintsev, Olah, and Tyka, “Inceptionism: Going Deeper into Neural Networks”, Google Research Blog . Images are licensed under CC-BY 4.0Choose an image and a layer in a CNN; repeat:1.Forward: compute activations at chosen layer 2.Set gradient of chosen layer equal to its activation 3.Backward: Compute gradient on image 4.Update imageCode is very simple but it uses a couple tricks: (Code is licensed under Apache 2.0)Code is very simple but it uses a couple tricks: (Code is licensed under Apache 2.0)Jitter imageCode is very simple but it uses a couple tricks: (Code is licensed under Apache 2.0)Jitter imageL1 Normalize gradientsCode is very simple butit uses a couple tricks:(Code is licensed under Apache 2.0)Jitter imageL1 Normalize gradientsClip pixel valuesAlso uses multiscale processing for a fractal effect (not shown)Sky image is licensed under CC-BY SA 3.0Image is licensed under CC-BY 4.0Image is licensed under CC-BY 4.0Image is licensed under CC-BY 3.0Image is licensed under CC-BY 3.0Image is licensed under CC-BY 4.0Given a CNN feature vector for an image, find a new image that: -Matches the given feature vector-“looks natural” (image prior regularization)Mahendran and Vedaldi, “Understanding Deep Image Representations by Inverting Them”, CVPR 2015Given feature vectorFeatures of new imageTotal Variation regularizer (encourages spatial smoothness)Reconstructing from different layers of VGG-16Mahendran and Vedaldi, “Understanding Deep Image Representations by Inverting Them”, CVPR 2015Figure from Johnson, Alahi, and Fei-Fei, “Perceptual Losses for Real-Time Style Transfer and Super-Resolution”, ECCV 2016. Copyright Springer, 2016.Reproduced for educational purposes.。

惯用语的翻译
• 有些习惯用语带有很浓的中国文化色彩， 有的含有中国历史典故或者古代人名，有 的涉及中国地名或宗教，若按照字面直译， 会让英美读者感到费解。
惯用语的翻译
• 赔了夫人又折兵 give one’s enemy a wife and lose one’s soldiers as well • (原文是三国典故，可改译成suffer a double loss instead of making a gain) • 跑了和尚跑不了庙 The monk may run away, but the temple can’t run with him. • (原文含有东方宗教色彩，可改译成A fugitive must belong to some place that can provide clues.)
Lecture 12
Chen weijuan
惯用语的翻译
• 惯用语是口头上常用的洗练、含蓄的语言， 比一般词和词组更具定型性，比四字成语 更具灵活性，比书面语更具讽刺、生动形 象的比喻性。惯用语主要出现在日常口语 中，书面上出现在小说、电影、电视、戏 剧的人物对话里，也出现在比较口语化的 议论文和散文中。由此可见，无论是口译， 还是笔译，都要涉及到惯较文气的惯用语和较口语化的四 字成语，但那毕竟是极少数。我们是从整 体上来比较这两者的翻译的。在具体翻译 时，我们一方面要记住惯用语的特点以及 它同四字成语在语体上的区别，另一方面 也要考虑到具体上下文来恰如其分地处理 其中的惯用语。
邓颖超致宋美龄的信（节选）
（1988年5月） • 庐山初识，忽忽五十年矣。山城之聚，金陵之晤，犹 历历如昨。别后音问阔绝四十余年，诚属憾事。幸友谊犹 存，两心相通。每遇客从远方来，道及夫人起居，更引起 怀旧之情。近闻夫人康健如常，颇感欣慰。 • 令姊孙夫人生前，极为思念夫人。每每言及夫人爱国 情切，必将致力于中国之统一。孙夫人手足情深，亟盼生 前能与夫人一晤；曾亲笔致函夫人，然未能如愿，终成遗 憾。 尝读夫人之《我将再起》，思感殊多。回首当年， 国难方殷，夫人致力全民抗战，促成国内团结，争取国际 援助，弘扬抗日民气，救助难童伤兵，厥功至伟。今夫人 年事虽高，犹时时关心国家之强盛，民族之再起，于海峡 两岸关系之缓和，亦多所推动。 夫人谋国之忠，诚如令 姊孙夫人所言，我极为钦佩！