1
Download Ciencias de Datos y Deep Learning: Neuronas artificiales para
Document related concepts
no text concepts found
Transcript
CienciasdeDatosy DeepLearning: Neuronasartificiales paraaprender Francisco Herrera Ciencia deDatosyDeepLearning: Neuronasartificialesparaaprender History of Data Science BigData 2 DeepLearning Ciencia deDatosyDeepLearning: Neuronasartificialesparaaprender RedesNeuronales En1943WarrenMcCullochyWalterPitts presentaronsumodelodeneurona artificial,ydescribieronlosprimerosfundamentosdeloquesellamaría posteriormenteredesneuronales. W.McCullochandW.Pitts(1943).ALogicalCalculusofideasImmanentinNervousActivity.BulletinofMathematicalBiophysics5:115-133. 3 Ciencia deDatosyDeepLearning: Neuronasartificialesparaaprender x1 x2 xn RedesNeuronales wj1 wj2 wjn n Netj yj Net j = w j1 × x1 + w j 2 × x2 + ! + w jn × xn = å w ji × xi i =1 æ ö y j = f ( Net j - q j ) = f çç å w ji × xi - q ÷÷ è ø Unaredneuronalseproponecomounsistemainteligentequeimitaalsistema nerviosoyalaestructuradelcerebro,peromuydiferenteentérminosdesu estructurayescala. Aligualquelasneuronasbiológicas,lasneuronasartificialesseinterconectan paraformarredesdeneuronasartificiales. Cadaneuronaartificialutilizaunafunciónprocesamiento queagregala informacióndeconexionesdeentradaconotrasneuronalesartificiales,una funcióndeactivaciónyunafuncióndetransferenciaparadarunasalidadela neuronaensusconexionesdesalida. 4 Ciencia deDatosyDeepLearning: Neuronasartificialesparaaprender RedNeuronalClásica:Backpropagation Credits:TheEvolutionofNeuralLearningSystems:ANovelArchitectureCombiningtheStrengthsofNTs,CNNs,and ELMs.NMartinel, CMicheloni…- IEEESMCMagazine,2015- ieeexplore.ieee.org 5 Ciencia deDatosyDeepLearning: Neuronasartificialesparaaprender DeepArchitecture(Redes Neuronales conmuchas capas) • Múltiples capasocultas • Aprendizajejerárquico • Característicascadavezmáscomplejas • Muybuencomportamiento enmútiplesdominios:Vision,Audio,… 6 Ciencia deDatosyDeepLearning: Neuronasartificialesparaaprender UnsupervisedDeepArchitecture: Autoencoder An autoencoder neuralnetworkis anunsupervisedlearning algorithmthatapplies backpropagation,settingthe targetvaluestobeequaltothe inputs. Theaimofanautoencoder isto learnarepresentation(encoding) forasetofdata,typicallyforthe purposeofdimensionality reduction. 7 Unsupervised feature learning with a neural network x1 x1 x2 x2 x3 x4 x5 x6 +1 a1 a2 a3 +1 Layer2 x3 x4 x5 x6 Layer3 Layer1 Andrew Ng Unsupervised feature learning with a neural network x1 x2 x3 x4 x5 x6 +1 a1 a2 a3 +1 New representation for input. Layer2 Layer1 Andrew Ng Ciencia deDatosyDeepLearning: Neuronasartificialesparaaprender DeepArchitecture: Autoencoder n BreastCancerWisconsin(Diagnostic)DataSet Autoencoder (f(x)=x) (unasolacapainternade3neuronasy1000 "epochs".WDBC(569instanciascon32atributosdeentrada) 10 Credito:D.Charte Ciencia deDatosyDeepLearning: Neuronasartificialesparaaprender DeepMind:Start up-2011 Demis Hassabis,Shane Legg yMustafa Suleyman 11 http://arxiv.org/abs/1312.5602 Ciencia deDatosyDeepLearning: Neuronasartificialesparaaprender JuegosArcade (Breakout) 12 Ciencia deDatosyDeepLearning: Neuronasartificialesparaaprender DeepLearning Retos enla“pintura” 13 Ciencia deDatosyDeepLearning: Neuronasartificialesparaaprender DeepLearning: DeepART 14 http://www.deepart.io/ Ciencia deDatosyDeepLearning: Neuronasartificialesparaaprender Ejemplos delresultado deDeepART vanGoth 15 http://www.deepart.io/ Ciencia deDatosyDeepLearning: Neuronasartificialesparaaprender Ejemplos delresultado deDeepART 16 http://www.deepart.io/ Ciencia deDatosyDeepLearning: Neuronasartificialesparaaprender ModelodeDLutilizadoydescripcióndelametodología http://arxiv.org/abs/1508.06576 17 Ciencia deDatosyDeepLearning: Neuronasartificialesparaaprender Supervised DeepLearning:Convolutional NeuralNetworks http://parse.ele.tue.nl/cluster/2/CNNArchitecture.jpg 18 Ciencia deDatosyDeepLearning: Neuronasartificialesparaaprender Supervised DeepLearning:Convolutional NeuralNetworks Eachmoduleconsistsofaconvolutional layerandapoolinglayer. Typicallytriestocompresslargedata(images)intoasmallersetofrobust features,basedonlocalvariations. Basicconvolutioncanstillcreatemanyfeatures. CNNshavebeenfoundhighlyeffectiveandbeencommonlyusedin computervisionandimagerecognition. 19 DeepLearning Convolutional steps https://github.com/rasbt/python-machine-learning-book/blob/master/faq/difference-deep-and-normallearning/convolution.png DeepLearning DigitRecognizerandConvolutional NN http://neuralnetworksanddeeplearning.com/chap6.html Convolutional neuralnetworks usethreebasicideas:localreceptivefields,shared weights,andpooling. Localreceptive fields:Tobemoreprecise,eachneuroninthefirsthiddenlayer willbeconnectedtoasmallregionoftheinputneurons,say,forexample,a5×5 region,correspondingto25inputpixels.So,foraparticularhiddenneuron,we mighthaveconnectionsthatlooklikethis: DeepLearning DigitRecognizerandConvolutional NN http://neuralnetworksanddeeplearning.com/chap6.html Localreceptive fields: 24×24 neurons 28×28 input image DeepLearning DigitRecognizerandConvolutional NN http://neuralnetworksanddeeplearning.com/chap6.html Shared weights andbiases:thesame weightsandbiasforeachofthe24×24 hiddenneurons(sigmoide function) Themapfromtheinputlayertothehiddenlayerafeaturemap. DeepLearning DigitRecognizerandConvolutional NN http://neuralnetworksanddeeplearning.com/chap6.html Shared weights andbiases: In the example shown, there are 3 feature maps. If we have 20 feature maps that's a total of 20×26=520 parameters defining the convolutional layer. By comparison, suppose we had a fully connected first layer, with 784=28×28 input neurons, 30 hidden neurons, 23,550 parameters. DeepLearning DigitRecognizerandConvolutional NN http://neuralnetworksanddeeplearning.com/chap6.html Pooling layers: thepoolinglayersdoissimplifytheinformationintheoutput fromtheconvolutional layer,onecommonprocedureforpoolingisknownasmaxpooling,inthe2x2regioninput. DeepLearning DigitRecognizerandConvolutional NN Pooling layers: DeepLearning DigitRecognizerandConvolutional NN http://neuralnetworksanddeeplearning.com/chap6.html The20imagescorrespondto20differentfeaturemaps Ciencia deDatosyDeepLearning: Neuronasartificialesparaaprender DeepLearning:MNISTdata Alaizquierda,losdígitosdeentradasinprocesar. Aladerecha,representacionesgráficasdelascaracterísticas 28 aprendidas.Enesencia,laredaprendea"ver"líneasybucles. Credits:https://www.datarobot.com/blog/a-primer-on-deep-learning/ DeepLearning DigitRecognizerandpreprocessing Asnapshotofimagepre-processingforconvolutional neuralnetworks:casestudy ofMNIST Siham Tabik,DanielPeralta,AndrésHerrera-Poyatos,FranciscoHerrera InternationalJournal ofComputational Intelligence Systems,Vol.10(2017)555–568 99.72accuracy CNNmodels:LeNet,Network3,DropConnet Preprocessing andaugmentation Ensembles DeepLearning DigitRecognizerandpreprocessing CNNmodels:LeNet [4] [4]Yann LeCun,Leon Bottou,Yoshua Bengio,andPatrickHaffner.Gradient-basedlearning appliedtodocumentrecognition.Proc.IEEE,86(11):2278– 2324,1998. DeepLearning DigitRecognizerandpreprocessing CNNmodels:Network3[13] [13]MichaelANielsen.Neuralnetworksanddeep learning.URL: http://neuralnetworksanddeeplearning.com,2015. DeepLearning DigitRecognizerandpreprocessing CNNmodels:DropConnet [14] [14]LiWan,Matthew Zeiler,Sixin Zhang,Yann LCun,andRobFergus.Regularizationof neuralnetworksusingdropconnect.InProceedingsofthe30thInternationalConference onMachineLearning(ICML- 13),pages 1058–1066,2013. DeepLearning DigitRecognizerandpreprocessing Preprocessing andaugmentation Translation.Theimageistranslatedanumberofpixelstowarda givendirection. Centering.Toeliminatewhitecolumns/rows,andtoresize by scaling. Rotation.Theimageisrotatedtoagivenangleθ. Elastic deformation.Image pixels areslightly movedinrandom directions,keepingtheimage’stopology. DeepLearning DigitRecognizerandpreprocessing Preprocessing andaugmentation DeepLearning DigitRecognizerandpreprocessing Preprocessing andaugmentation DeepLearning DigitRecognizerandpreprocessing Preprocessing andaugmentation DeepLearning DigitRecognizerandpreprocessing Preprocessing andaugmentation DeepLearning DigitRecognizerandpreprocessing Preprocessing andaugmentation DeepLearning DigitRecognizerandpreprocessing Preprocessing andaugmentation DeepLearning DigitRecognizerandpreprocessing Preprocessing andaugmentation DeepLearning DigitRecognizerandpreprocessing Preprocessing andaugmentation Ensembles DeepLearning DigitRecognizerandpreprocessing Preprocessing andaugmentation The28handwritten digitsmisclassifiedby ensemble-5of Network3 Thedigitbetween() representsthecorrect class. The13digitslabeled withasterisksarealso misclassifiedby DropConnect Deep Learning: MNIST data Asnapshotofimagepre-processingforconvolutional neuralnetworks:casestudy ofMNIST Siham Tabik,DanielPeralta,AndrésHerrera-Poyatos,FranciscoHerrera InternationalJournal ofComputational Intelligence Systems,Vol.10(2017)555–568 99.72accuracy The28handwritten digitsmisclassifiedby ensemble-5of DropConnet Thedigitbetween() representsthecorrect class. The13digitslabeled withasterisksarealso misclassifiedby Network3 DeepLearning DigitRecognizerandpreprocessing Preprocessing andaugmentation The13handwritten digitsmisclassifiedby ensemble-5of DropConnet andNetwork3 DeepLearning MXNet http://mxnet.io/get_started/index.html DeepLearning MXNet http://mxnet.io/how_to/finetune.html DeepLearning Learningfromscratchvs fine-tuning(GoogLeNet yVGG-16) VGG16 http://www.robots.ox.ac.uk/~vgg/research/very_deep/ K.Simonyan,A.Zisserman VeryDeepConvolutional NetworksforLarge-ScaleImageRecognition arXiv technicalreport,2014 https://arxiv.org/pdf/1409.1556.pdf VGGisaconvolutional neuralnetworkmodelproposedbyK.Simonyan andA.Zisserman fromtheUniversityofOxfordinthepaper“VeryDeepConvolutional NetworksforLargeScaleImageRecognition”.Themodelachieves92.7%top-5testaccuracyinImageNet , whichisadatasetofover14millionimagesbelongingto1000classes. https://gist.github.com/ksimonyan/211839e770f7b538e2d8#file-readme-md ##Information name:16-layermodelfromthearXiv paper:"VeryDeepConvolutional NetworksforLargeScaleImageRecognition" caffemodel:VGG_ILSVRC_16_layers caffemodel_url: http://www.robots.ox.ac.uk/~vgg/software/very_deep/caffe/VGG_ILSVRC _16_layers.caffemodel license:see http://www.robots.ox.ac.uk/~vgg/research/very_deep/ caffe_version:trainedusingacustomCaffe-basedframework DeepLearning Learningfromscratchvs fine-tuning(GoogLeNet yVGG-16) VGG16 Source:https://www.cs.toronto.edu/~frossard/post/vgg16/ DeepLearning Learningfromscratchvs fine-tuning(GoogLeNet yVGG-16) VGG16 Modelandpre-trainedparametersforVGG16inTensorFlow https://www.cs.toronto.edu/~frossard/post/vgg16/ Keras:https://keras.io/ Models for image classification https://keras.io/applications/ with weights trained on ImageNet DeepLearninglibraryforTheano andTensorFlow Keras isahigh-levelneuralnetworksAPI,writtenin Pythonandcapableofrunningontopof either TensorFlow or Theano.Itwasdevelopedwitha focusonenablingfastexperimentation. Beingabletogo fromideatoresultwiththeleastpossibledelayiskeyto doinggoodresearch. DeepLearning Learningfromscratchvs fine-tuning(GoogLeNet yVGG-16) GoogLeNet https://research.google.com/pubs/pub43022.html Weproposeadeepconvolutional neuralnetworkarchitecturecodenamedInceptionthatachievesthe newstateoftheartforclassificationanddetectionintheImageNet Large-ScaleVisualRecognition Challenge2014(ILSVRC2014).Themainhallmarkofthisarchitectureistheimprovedutilizationofthe computingresourcesinsidethenetwork.Byacarefullycrafteddesign,weincreasedthedepthandwidth ofthenetworkwhilekeepingthecomputationalbudgetconstant.Tooptimizequality,thearchitectural decisionswerebasedontheHebbian principleandtheintuitionofmulti-scaleprocessing.Oneparticular incarnationofthisarchitecture,GoogLeNet,a22layersdeepnetwork,wasusedtoassessitsqualityin thecontextofobjectdetectionandclassification. Going Deeper with Convolutions ChristianSzegedy, Wei Liu, Yangqing Jia, PierreSermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, Andrew Rabinovich arXiv technicalreport,2014 DeepLearning Learningfromscratchvs fine-tuning(GoogLeNet yVGG-16) GoogLeNet http://www.robots.ox.ac.uk/~vgg/research/very_deep/ GooglLeNet inCaffe https://github.com/BVLC/caffe/tree/master/models/bvlc_googlenet GooglLeNet inKeras https://gist.github.com/joelouismarino/a2ede9ab3928f999575423b9887abd14 http://joelouismarino.github.io/blog_posts/blog_googlenet_keras.html Ciencia deDatosyDeepLearning: Neuronasartificialesparaaprender Deep Learning: Detección de Armas en Video R. Olmos, S. Tabik, F. Herrera (UGR) 52 Video:Skyfall,2012 Ciencia deDatosyDeepLearning: Neuronasartificialesparaaprender Deep Learning: Detección de Armas en Video R. Olmos, S. Tabik, F. Herrera (UGR) 53 Video:Skyfall,2012 Ciencia deDatosyDeepLearning: Neuronasartificialesparaaprender Ejemplosdeaplicaciones Fingerprintidentification/classification Cellclassificationfromcervixsmears Underwaterimagepatchesofcorals 54 Ciencia deDatosyDeepLearning: Neuronasartificialesparaaprender DeepLearning Lectura:RecentOverview 55 http://www.nature.com/nature/journal/v521/n7553/full/nature14539.html Ciencia deDatosyDeepLearning: Neuronasartificialesparaaprender LeCum,Bengio yHinton,2015:“Ultimately,majorprogressin artificialintelligencewillcomeaboutthroughsystemsthatcombine representationlearningwithcomplexreasoning.Althoughdeep learningandsimplereasoninghavebeenusedforspeechand handwritingrecognitionforalongtime,newparadigmsareneeded toreplacerule-basedmanipulationofsymbolicexpressionsby operationsonlargevectors” Losavancesenelconocimientodelcerebroyelrazonamiento humanopermitirándiseñarnuevosparadigmasde representaciónyrazonamientocomplejo. 56
