Gerald Friedland, Adjunct Assistant Professor

Closed (1) Learning Multimodal Joint Representation from Multimedia Big Data

Closed. This professor is continuing with Fall 2018 apprentices on this project; no new apprentices needed for Spring 2019.

We are looking for a few undergraduate researchers interested in building a multimedia retrieval system using joint representation learned from multimodal data (in our case, videos from YouTube or Flickr).
One possible goal of this project would be a winning submission to next round of YouTube-8M Video Understanding Challenge (this year's challenge just ended).

Related publications:
- Sampled Image Tagging and Retrieval Methods on User Generated Content (
- See, Hear, and Read: Deep Aligned Representations (
- Deep Neural Architecture for Multi-Modal Retrieval based on Joint Embedding Space for Text and Images (
- Tutorial on Multimodal Machine Learning:

Undergraduate researcher will learn to design and implement a system using tools and machine learning frameworks used in the state-of-the-art research in Multimedia Computing.

The student is expected to meet with mentor at least once a week and attend weekly lab meeting at which we will discuss the results and direction of ongoing experiments. In addition, the apprentice will be responsible for reading several primary research articles and presenting them to the research group, in order to increase his/her familiarity with the field.

The emphasis of this position will be on exposing the apprentice to the experimental research process through hands on experience, and developing skills and knowledge of techniques that may be used in future academic and research endeavors. Successful research apprentices complete their assignments in a timely manner, maintain open communication with other members of the research group and with the research coordinator, ask questions when they need help or guidance, and actively ensure (through communicating with the research coordinator) that they are getting the experience they want from the URAP program.

Day-to-day supervisor for this project: Jaeyoung Choi, Ph.D. candidate

Qualifications: Major is preferably computer science, statistics or any other science connected to large scale data analysis. - Previous experience and/or coursework in machine learning is required (CS189 or equivalent) - We have a strong preference for juniors or seniors who can participate for at least one year. - GPA of at least 3.5 - Required Skill: Python, Linux, Tensorflow or Pytorch - You should be able to understand the code and follow the materials discussed in the following page: - (*IMPORTANT*) At least 15 hours per week time commitment required for this project.

Weekly Hours: 12 or more hours

Related website:

Closed (2) Machine Learning, Physics and Information Theory

Check back for status

This is a strongly multidisciplinary research project, involving AI,
information theory/signal processing and physics. The goal is to develop
algorithms and methods that bridge the fields for the purpose of
integrating machine learning with science. This allows the use of
machine learning as part of physical, chemical and biological
simulations while at the same time fostering a better understanding of AI.
Past results in our research include this demo:

The candidate will:
a) conduct rigorous scientific research in practice
b) develop ideas into publications
c) software engineering skills

Task 1: AI/Machine Learning research
Sub-Task 1a: Develop a data-independent benchmarking for popular AI frameworks, like Keras, Tensorflow, Theano, Torch etc.
Sub-Task 1b: Develop methods to automatically design neural networks based on user input data sets.

Task 2: Physics/Information Theory research
Sub-Task: 2a: Use channel capacity on conservative and non-conservative Hamiltonian systems, with periodic, quasiperiodic and chaotic orbits. Analyze and report the differences.
Sub-task 2b: Generate molecular dynamics trajectory data for different systems. Integrate channel capacity measurement in a LAMMPS script
Outcome: A scientific paper documenting the LAMMPS script able to measure channel capacity on-the-fly for any simulation.

Day-to-day supervisor for this project: Alfredo Metere, Staff Researcher

Qualifications: - Be eat least a junior. - Required majors: computer science, electrical engineering, physics, chemistry, data science. - The applicant is required to be open-minded and demonstrate independent, rational thinking without fear of expressing his/her own opinion, motivating it, even if in opposition to the supervisor. - The applicants for the AI path are required to have attended the basic machine learning course (Data 8 and Data 100). It is desirable but not essential to have visited one advanced course in machine learning. - The applicants for the physics path are required to have passed the following courses or demonstrate the possession of analogous core competences: - PHYSICS 77 - PHYSICS 151 - E 40-Engineering Thermodynamics or Physics 112-Introduction to Statistical and Thermal Physics - ME 104-Engineering Mechanics or Physics 105-Analytic Mechanics and have at least a basic knowledge about nonlinear dynamical systems. - For the each task/sub-task, the undergraduate is encouraged to exchange ideas with the supervisor. - Applicants without a clear reference in their application to this project (Machine Learning, Physics and/or Information Theory) or not commenting on the required qualification will not be considered. - A student can propose to cover tasks across the disciplines, for a maximum total of two tasks per student. Such proposal should be supported by written motivation. - Required: C, Python, Linux, Intro to Artificial Intelligence - The candidates will be mainly screened for their capacity in independent, critical thinking and curiosity. - Depth of understanding will be prioritized over volume of knowledge or trial and error.

Weekly Hours: 12 or more hours

Related website:
Related website: