SciCombinator

Discover the most talked about and latest scientific content & concepts.

33

The theory of reinforcement learning provides a normative account, deeply rooted in psychological and neuroscientific perspectives on animal behaviour, of how agents may optimize their control of an environment. To use reinforcement learning successfully in situations approaching real-world complexity, however, agents are confronted with a difficult task: they must derive efficient representations of the environment from high-dimensional sensory inputs, and use these to generalize past experience to new situations. Remarkably, humans and other animals seem to solve this problem through a harmonious combination of reinforcement learning and hierarchical sensory processing systems, the former evidenced by a wealth of neural data revealing notable parallels between the phasic signals emitted by dopaminergic neurons and temporal difference reinforcement learning algorithms. While reinforcement learning agents have achieved some successes in a variety of domains, their applicability has previously been limited to domains in which useful features can be handcrafted, or to domains with fully observed, low-dimensional state spaces. Here we use recent advances in training deep neural networks to develop a novel artificial agent, termed a deep Q-network, that can learn successful policies directly from high-dimensional sensory inputs using end-to-end reinforcement learning. We tested this agent on the challenging domain of classic Atari 2600 games. We demonstrate that the deep Q-network agent, receiving only the pixels and the game score as inputs, was able to surpass the performance of all previous algorithms and achieve a level comparable to that of a professional human games tester across a set of 49 games, using the same algorithm, network architecture and hyperparameters. This work bridges the divide between high-dimensional sensory inputs and actions, resulting in the first artificial agent that is capable of learning to excel at a diverse array of challenging tasks.

Concepts: Psychology, Artificial intelligence, Machine learning, Learning, Neural network, Dopamine, Reinforcement learning, Atari 2600

28

Hypothalamic pro-opiomelanocortin (POMC) neurons promote satiety. Cannabinoid receptor 1 (CB1R) is critical for the central regulation of food intake. Here we test whether CB1R-controlled feeding in sated mice is paralleled by decreased activity of POMC neurons. We show that chemical promotion of CB1R activity increases feeding, and notably, CB1R activation also promotes neuronal activity of POMC cells. This paradoxical increase in POMC activity was crucial for CB1R-induced feeding, because designer-receptors-exclusively-activated-by-designer-drugs (DREADD)-mediated inhibition of POMC neurons diminishes, whereas DREADD-mediated activation of POMC neurons enhances CB1R-driven feeding. The Pomc gene encodes both the anorexigenic peptide α-melanocyte-stimulating hormone, and the opioid peptide β-endorphin. CB1R activation selectively increases β-endorphin but not α-melanocyte-stimulating hormone release in the hypothalamus, and systemic or hypothalamic administration of the opioid receptor antagonist naloxone blocks acute CB1R-induced feeding. These processes involve mitochondrial adaptations that, when blocked, abolish CB1R-induced cellular responses and feeding. Together, these results uncover a previously unsuspected role of POMC neurons in the promotion of feeding by cannabinoids.

Concepts: Receptor, Opioid, Receptor antagonist, Opioid receptor, Cannabinoid receptor, Dopamine, Buprenorphine, Proopiomelanocortin

28

Long-term in vivo expression of a broad and potent entry inhibitor could circumvent the need for a conventional vaccine for HIV-1. Adeno-associated virus (AAV) vectors can stably express HIV-1 broadly neutralizing antibodies (bNAbs). However, even the best bNAbs neutralize 10-50% of HIV-1 isolates inefficiently (80% inhibitory concentration (IC80) > 5 μg ml(-1)), suggesting that high concentrations of these antibodies would be necessary to achieve general protection. Here we show that eCD4-Ig, a fusion of CD4-Ig with a small CCR5-mimetic sulfopeptide, binds avidly and cooperatively to the HIV-1 envelope glycoprotein (Env) and is more potent than the best bNAbs (geometric mean half-maximum inhibitory concentration (IC50) < 0.05 μg ml(-1)). Because eCD4-Ig binds only conserved regions of Env, it is also much broader than any bNAb. For example, eCD4-Ig efficiently neutralized 100% of a diverse panel of neutralization-resistant HIV-1, HIV-2 and simian immunodeficiency virus isolates, including a comprehensive set of isolates resistant to the CD4-binding site bNAbs VRC01, NIH45-46 and 3BNC117. Rhesus macaques inoculated with an AAV vector stably expressed 17-77 μg ml(-1) of fully functional rhesus eCD4-Ig for more than 40 weeks, and these macaques were protected from several infectious challenges with SHIV-AD8. Rhesus eCD4-Ig was also markedly less immunogenic than rhesus forms of four well-characterized bNAbs. Our data suggest that AAV-delivered eCD4-Ig can function like an effective HIV-1 vaccine.

Concepts: HIV, Immune system, Microbiology, Macaque, Primate, Rhesus Macaque, Adeno-associated virus, Neutralization

26

The Martian limb (that is, the observed ‘edge’ of the planet) represents a unique window into the complex atmospheric phenomena occurring there. Clouds of ice crystals (CO2 ice or H2O ice) have been observed numerous times by spacecraft and ground-based telescopes, showing that clouds are typically layered and always confined below an altitude of 100 kilometres; suspended dust has also been detected at altitudes up to 60 kilometres during major dust storms. Highly concentrated and localized patches of auroral emission controlled by magnetic field anomalies in the crust have been observed at an altitude of 130 kilometres. Here we report the occurrence in March and April 2012 of two bright, extremely high-altitude plumes at the Martian terminator (the day-night boundary) at 200 to 250 kilometres or more above the surface, and thus well into the ionosphere and the exosphere. They were spotted at a longitude of about 195° west, a latitude of about -45° (at Terra Cimmeria), extended about 500 to 1,000 kilometres in both the north-south and east-west directions, and lasted for about 10 days. The features exhibited day-to-day variability, and were seen at the morning terminator but not at the evening limb, which indicates rapid evolution in less than 10 hours and a cyclic behaviour. We used photometric measurements to explore two possible scenarios and investigate their nature. For particles reflecting solar radiation, clouds of CO2-ice or H2O-ice particles with an effective radius of 0.1 micrometres are favoured over dust. Alternatively, the plume could arise from auroral emission, of a brightness more than 1,000 times that of the Earth’s aurora, over a region with a strong magnetic anomaly where aurorae have previously been detected. Importantly, both explanations defy our current understanding of Mars' upper atmosphere.

25

Glycyrrhiza glabra (licorice) has been known to possess various pharmacological properties including anti-inflammatory, antioxidants, antiviral, and hepatoprotective activities. Magnesium isoglycyrrhizinate (MgIG), a magnesium salt of 18-α glycyrrhizic acid stereoisomer, is clinically used for the treatment of inflammatory liver diseases. However, the mechanism by which MgIG exerts its anti-inflammatory effects remains unknown. In the present study, we investigated the inhibitory potential of MgIG in phospholipase A2 (PLA2)/arachidonic acid (AA) pathway and release of the pathway-generated inflammatory lipid mediators in RAW264.7 macrophages. Results revealed that MgIG suppressed LPS-induced activation of PLA2 and production of AA metabolites such as prostaglandin E2 (PGE2), prostacyclin (PGI2), thromboxane 2 (TXB2), and leukotrienes (LTB4) in macrophages. Furthermore, LPS-induced AA-metabolizing enzymes including COX-2, COX-1, 5-LOX, TXB synthase, and PGI2 synthase were significantly inhibited by MgIG. Taken together, our data suggest that modulation of cyclooxygenase (COXs) and 5-lipoxygenase (LOX) pathways in AA metabolism could be a novel mechanism for the anti-inflammatory effects of MgIG.

Concepts: Inflammation, Metabolism, Cyclooxygenase, Phospholipase A2, Prostacyclin, Prostaglandin, Arachidonic acid, Liquorice

22

So far, roughly 40 quasars with redshifts greater than z = 6 have been discovered. Each quasar contains a black hole with a mass of about one billion solar masses (10(9) ). The existence of such black holes when the Universe was less than one billion years old presents substantial challenges to theories of the formation and growth of black holes and the coevolution of black holes and galaxies. Here we report the discovery of an ultraluminous quasar, SDSS J010013.02+280225.8, at redshift z = 6.30. It has an optical and near-infrared luminosity a few times greater than those of previously known z > 6 quasars. On the basis of the deep absorption trough on the blue side of the Lyman-α emission line in the spectrum, we estimate the proper size of the ionized proximity zone associated with the quasar to be about 26 million light years, larger than found with other z > 6.1 quasars with lower luminosities. We estimate (on the basis of a near-infrared spectrum) that the black hole has a mass of ∼1.2 × 10(10) , which is consistent with the 1.3 × 10(10) derived by assuming an Eddington-limited accretion rate.

Concepts: Galaxy, Big Bang, Redshift, General relativity, Mass, Universe, Quasar, Gravitational lens

22

The question of whether animals have emotions and respond to the emotional expressions of others has become a focus of research in the last decade [1-9]. However, to date, no study has convincingly shown that animals discriminate between emotional expressions of heterospecifics, excluding the possibility that they respond to simple cues. Here, we show that dogs use the emotion of a heterospecific as a discriminative cue. After learning to discriminate between happy and angry human faces in 15 picture pairs, whereby for one group only the upper halves of the faces were shown and for the other group only the lower halves of the faces were shown, dogs were tested with four types of probe trials: (1) the same half of the faces as in the training but of novel faces, (2) the other half of the faces used in training, (3) the other half of novel faces, and (4) the left half of the faces used in training. We found that dogs for which the happy faces were rewarded learned the discrimination more quickly than dogs for which the angry faces were rewarded. This would be predicted if the dogs recognized an angry face as an aversive stimulus. Furthermore, the dogs performed significantly above chance level in all four probe conditions and thus transferred the training contingency to novel stimuli that shared with the training set only the emotional expression as a distinguishing feature. We conclude that the dogs used their memories of real emotional human faces to accomplish the discrimination task.

Concepts: Psychology, Emotion, Discrimination, Affect display, Emotions and culture, Emotional expression, Racial discrimination, Rod Stewart

21

Darwin’s finches, inhabiting the Galápagos archipelago and Cocos Island, constitute an iconic model for studies of speciation and adaptive evolution. Here we report the results of whole-genome re-sequencing of 120 individuals representing all of the Darwin’s finch species and two close relatives. Phylogenetic analysis reveals important discrepancies with the phenotype-based taxonomy. We find extensive evidence for interspecific gene flow throughout the radiation. Hybridization has given rise to species of mixed ancestry. A 240 kilobase haplotype encompassing the ALX1 gene that encodes a transcription factor affecting craniofacial development is strongly associated with beak shape diversity across Darwin’s finch species as well as within the medium ground finch (Geospiza fortis), a species that has undergone rapid evolution of beak shape in response to environmental changes. The ALX1 haplotype has contributed to diversification of beak shapes among the Darwin’s finches and, thereby, to an expanded utilization of food resources.

Concepts: DNA, Genetics, Evolution, Species, Charles Darwin, Darwin's finches, Finch, Medium Ground-finch

20

The Black Death, originating in Asia, arrived in the Mediterranean harbors of Europe in 1347 CE, via the land and sea trade routes of the ancient Silk Road system. This epidemic marked the start of the second plague pandemic, which lasted in Europe until the early 19th century. This pandemic is generally understood as the consequence of a singular introduction of Yersinia pestis, after which the disease established itself in European rodents over four centuries. To locate these putative plague reservoirs, we studied the climate fluctuations that preceded regional plague epidemics, based on a dataset of 7,711 georeferenced historical plague outbreaks and 15 annually resolved tree-ring records from Europe and Asia. We provide evidence for repeated climate-driven reintroductions of the bacterium into European harbors from reservoirs in Asia, with a delay of 15 ± 1 y. Our analysis finds no support for the existence of permanent plague reservoirs in medieval Europe.

Concepts: Infectious disease, Middle Ages, Europe, Pandemic, Italy, Yersinia pestis, Black Death, Silk Road

12

We present a pinned insect manipulator (IMp) constructed of LEGO® building bricks with two axes of movement and two axes of rotation. In addition we present three variants of the IMp to emphasise the modular design, which facilitates resizing to meet the full range of pinned insect specimens, is fully customizable, collapsible, affordable and does not require specialist tools or knowledge to assemble.

Concepts: Design, Rotation, IMP