Category: Students

Categories
Featured Students

AI Month, NSF Fellowship, & Enhancing Language Models

In celebration of AI Month, we are thrilled to extend our congratulations to Andrew Zhu, under the guidance of Chris Callison-Burch, for his remarkable achievement in receiving the prestigious 2024 NSF Graduate Research Fellowship Program (GRFP). Andrew’s dedication and innovative vision have earned him this esteemed recognition, marking a significant milestone in his academic journey.

Andrew’s research endeavors are poised to address a critical challenge in the realm of Language Model Machines or Large Language Models (LLMs), where despite their formidable capabilities, they often grapple with the issue of ‘forgetting’ when confronted with complex inquiries or tasks spanning extensive timelines. For instance, while humans effortlessly navigate through multi-step queries, LLMs encounter hurdles in retaining the original context across successive stages of information retrieval. Inspired by this gap, Andrew’s previous work, FanOutQA, shed light on the limitations of current models, showcasing the vast barrier between human accuracy and machine performance.

High-level diagram showing the settings in the FanOutQA benchmark: the LLM (robot) must answer a complex question by breaking it down (bottom). There are 3 settings we test LLMs in: closed-book (the LLM cannot use any search tools, only what it was trained on), open-book (the LLM may search Wikipedia), and evidence-provided (the LLM is given the Wikipedia pages containing the answers and must extract the correct answers).

Central to Andrew’s proposed solution is the concept of recursive delegation, a groundbreaking approach aimed at endowing LLMs with the ability to allocate intricate tasks to subordinate LLMs. This hierarchical structure allows for the decomposition of complex problems into more manageable segments, with each ‘child’ LLM equipped to handle its designated subtask. Through this cascading delegation, Andrew envisions a synergistic collaboration among LLMs, comparable to a relay race where each participant contributes to achieving the final objective.

In laying the groundwork for his research, Andrew has developed Kani, a versatile framework designed to facilitate seamless interaction between LLMs and human-written code. This innovative tool empowers researchers to experiment with different LLM architectures while interfacing with Python functions effortlessly. By democratizing access to such tools, Andrew aims to catalyze progress in the field and foster a collaborative ecosystem of exploration and discovery.

Screenshot of an early proof-of-concept recursive delegation system I developed this year. In this example, it’s helping make travel plans for a Japan trip and looking at Shinkansen (bullet train) travel times on the internet. Each node (in the top right) is an independent LLM. (green=in progress, yellow=waiting, gray=completed, blue=selected)
The same as above, but with the code needed to actually use Kani. It’s designed to be easy to pick up while allowing experienced developers to customize the library extensively.
An overview of what the Kani system is comprised of. It manages the chat history and provides it and user-written functions to underlying LLMs (engines) with a common interface.

Andrew’s advisor, Chris Callison-Burch, effectively summarizes the significance of his work, stating,

“Andrew Zhu’s research is at the cutting edge of programming languages, natural language processing, and artificial intelligence. His groundbreaking open-source software package, Kani, elegantly solves a major weakness of large language models (LLMs) like ChatGPT by enabling them to write code and call functions. Kani seamlessly integrates LLMs into Python programs, allowing developers to write functions that can be called by the language model. This game-changing approach combines the strengths of AI and traditional programming, enabling LLMs to interweave natural language generation with complex computations.

Andrew’s work has the potential to revolutionize how we develop AI systems, paving the way for more powerful and versatile AI applications that go beyond ChatGPT’s current abilities of understanding and generating natural language to also include the abilities to create and manipulate code.”

Chris Callison-Burch

As Andrew embarks on this ambitious journey, his work holds the promise of transforming the landscape of artificial intelligence, ushering in a new era of collaboration and problem-solving. We eagerly anticipate the insights and advancements that will emerge from his pioneering efforts, underscoring the transformative potential of AI in shaping our collective future. Once again, congratulations to Andrew Zhu on this well-deserved recognition, and here’s to a future marked by innovation, collaboration, and boundless possibilities in the realm of AI.

References:

[1] Andrew Zhu, Alyssa Hwang, Liam Dugan, and Chris Callison-Burch. 2024. FanOutQA: Multi-Hop, Multi-Document Question Answering for Large Language Models. Arxiv preprint, in submission at ACL 2024. https://arxiv.org/abs/2402.14116

[2] Andrew Zhu, Liam Dugan, Alyssa Hwang, and Chris Callison-Burch. 2023. Kani: A Lightweight and Highly Hackable Framework for Building Language Model Applications. In Proceedings of the 3rd Workshop for Natural Language Processing Open Source Software (NLP-OSS 2023), pages 65–77, Singapore. Association for Computational Linguistics. https://aclanthology.org/2023.nlposs-1.8/

Categories
Students

The C.I.S. Blog Presents- “DALL-E Art Gallery”

University of Pennsylvania, by Beeple (prompt generated by Yuxin Meng)

So far in this semester, our department blog has talked about the growths in our labs and our faculty members careers. We have touched on the incredible seminars that have been taking place as well as the exciting research projects that our students are involved in. The blog’s overall purpose is to showcase the technological strides that Penn Engineering is making as well as the significant academic achievements of our faculty and student body. The C.I.S. blog is also a platform that strives to implement humanity and relatability to those who are a part of the Penn community and those outside of the University.

As many of you may know, DALL-E 2 developed by OpenAI was launched in April of this year and has just recently been made available to anyone. You get an allowance of free credits that enable you to type in a detailed description of anything that comes to mind and the machine learning models generate digital images that reflect the prompt. We were interested in the combination between artificial intelligence and the ability to generate realistic images using human ingenuity.

DALL-E is a great example of AI and human collaboration working to break barriers and expand horizons through artistic creativity. This platform also gives people the ability to play and be as out there and imaginative as they want. All in all, DALL-E gives us the opportunity to have fun and explore our hobbies, interests, and studies in the form of art. To showcase this AI system and what it can do I had asked all students from the C.I.S. department, from Undergrad to Ph.D., to send in the descriptions that they prompted and to have fun with it.

With that being said, this Gallery of DALL-E generated art was made possible by some of our wonderful students in the C.I.S. Department!

Ani Petrosyan, she/her
Computer Science major, 2026

Purple mountains with Armenian waterfalls

"I am an international student from Armenia. It's a very mountainous country, with waterfalls and wonderful nature. One fun fact about me: most of the dreams I see have purple color in them, so I am dreaming of my country and seeing it in purple.
A.
B.
Edward Hu, he/him
Ph.D. in Robot Learning

A. Bob Ross in the style of Picasso uncanny unreal engine

"Bob Ross is an iconic painter, so I would like AI painters to pay homage to him."

B. Darth Vader cooking in Hell's Kitchen

"Hell's Kitchen is one of my favorite shows. I think Darth Vader's past with high heat and pressure scenarios would make him an excellent contestant."
A.
B.
C.
Yuxin Meng, she/her
MCIT, 2024

A. Hacker, another dimension, digital art

"This one was intended to be "software engineer..." or "coding in..." but these keywords seemed to be less instructive compared to "hacker". I wanted to see us working on the same thing in another universe."

B. Bionic sheep, blueprints

"Love the book: Do Androids Dream of Electric Sheep? Electric sheep in the book look like real sheep, so I guess bionic is proper."

C. In lab, machine reading brain, codes on computer science, digital, by Beeple

"Not so much what I pictured in my mind. I've been obsessed with Pantheon (science fiction drama) recently. Briefly, I expected a picture of machine scanning human brain as code."
Rotem Dror, she/her
Postdoc in the Cognitive Computation Group

Two computers compete in a running competition

"I needed an image for my job talk presentation that would show two models competing who are going to be state of the art. My research involves developing statistical methods for comparing NLP models to determine which is better."
A.
B.
C.
D.
Hannah Gonzalez, she/her
MSE and BSE in Computer Science, 2023

A. A red fox surfing The Great Wave off Kanagawa by Katsushika Hokusai

B. Huskies sledding in Alaska by Monet

C. Macro 35mm film photography of a floating otter wearing a space suit with the Van Gogh "Starry Night" painted background

D. An Andy Warhol style painting of a corgi winking
Gaoxiang Luo, he/him
Ph.D. in Machine Learning

"I generated an image using my hobbies as keywords: cat, guitar, and latte art. I was very impressed and surprised that the AI considered the cat element as latte art!"
Categories
Featured Students

C.I. S. and Penn Engineering Halloween Celebration 2022

Penn Engineering’s Halloween, Fall 2022

This year’s Halloween celebration for Penn Engineering was a great success! Held in Quain Courtyard this past Monday, October 31, students, staff, and faculty gathered for trick or treating, hot apple cider and hot chocolate, and a costume contest. There was a great turn out of students coming in between classes and during their lunch break. In addition, incredibly decorated tables from each department covered the courtyard and there was wonderful staff participation. Even some faculty members came out in costume! It was a great experience interacting with students for some spooky and relaxing fun!

Scroll down to see the department tables and awesome costumes that came out for another fun year of Halloween festivities!

C.I.S. Department Chair and Staff
C.I.S. Department Director of Administrative Operations, Jackie Caliman, and student
MEAM
ESE
BE
MSE
CIS
CBE
Office of Diversity, Equity and Inclusion
Career Services
Academic Services
Categories
Students

Welcome Back C.I.S. Students: Let’s See What’s New!

Melvin J. and Claire Levine Hall

The 2022-2023 academic year has kicked off last week and summer has officially come to a close. We would like to welcome back returning Computer and Information Science students as well as the Class of 2026! Penn Engineering is excited to have you on board.

With a new school year comes new changes for Penn Engineering and the CIS department. In the past year, we have hired an exceptional number of faculty, brought in new research initiatives, renovated our spaces, and broke ground on a state-of-the-art facility. We are also incredibly thankful to be able to see so many faces in person this year as the circumstances surrounding Covid-19 continues to improve and in-person activities can commence.

Faculty

Several brand new assistant professors have joined the department this semester, while the rest will arrive in January and next Fall. Rejoining the CIS department, in a new role as an Assistant Professor, is Osbert Bastani who develops innovative techniques for programming and building software that incorporate machine learning components. Two new faces to Penn Engineering features Danaë Metaxa, who works in areas of human-computer interaction and communications, and Eric Wong, who works on robust and reliable machine learning.

“We are delighted to welcome an unprecedented 10 new assistant professors arriving over the next year.  Each brings new innovations to the curriculum and more opportunities to get involved in undergraduate research projects.”, says Zachary Ives, Chair of the CIS Department.

Each new faculty member entering into Computer and Information Science has arrived to break barriers and help our students grow.

Space

As our department is growing, our spaces have been transforming as well. This summer, the Distributed Systems Lab (DSL) and the SIG Lab for Computer Graphics on the first floor of Moore have both undergone major renovations. On the second floor of Levine Hall, the brand new Penn Human-Computer Interaction Lab, led by Andrew Head and Danaë Metaxa, just opened.

“We couldn’t be more excited for the start of this year and the official launch of our group. We’re looking forward to teaching Penn’s first Human-Computer Interaction courses at all levels, welcoming our first cohort of PhD students and opening our physical space- the HCI Lab- in Levine 255. We welcome interested students to reach out to us!” -Danaë Metaxa, Assistant Professor, CIS Dept, University of Pennsylvania.

In addition to our growth in space, Penn Engineering is not stopping there. The construction of Amy Gutmann Hall has begun during the summer. The creation of new space, study rooms, and research labs is anticipated to be opening in September 2024.

Research Initiatives

Since the launch of this past year’s interdisciplinary research initiative, Innovation in Data Engineering and Science (IDEAS), the CIS Department announced the ASSET Center directed by Rajeev Alur. ASSET (AI-enabled Systems: Safe, Explainable, Trustworthy) focuses on implementing tools and science to guarantee AI systems do exactly what they are designed to do. Getting students involved in this new initiative is a top priority for the Center.

“The best way to get involved is to join our seminars. It’s every Wednesday at noon and we have a great line-up of speakers. There is a number of faculty from our department, other Penn faculty, and also outside speakers. Some of the topics will include applications to healthcare, explainablility, and safety.” -Rajeev Alur, Director of ASSET, University of Pennsylvania.

With every new faculty member, space renovation, and research initiative; all of these things are implemented to give students the best opportunities for success. We have another exciting year just beginning at Penn Engineering. Let’s make it count.

Categories
Featured Students

Happy Halloween! From CIS and Penn Engineering 👻🎃🦇

On Friday, October 29, each department of the School of Engineering and Applied Science gathered together on the West Towne Lawn in the spirit of Halloween celebration!

Students, staff and faculty were able to stop by each department’s station for delicious treats and candy, Penn Engineering swag, and fun Halloween stickers and toys.

Not only were costumes encouraged, but the Penn Engineering community is hosting a costume contest, with entries accepted until November 3!

Scroll down for some amazing photos of the day, which also included a photo backdrop and Halloween tunes!

Other CIS-affiliated Halloween events include:

  • The Penn Society of Women Engineers Meet and Greet – Levine Lobby, October 29, 4-5pm
    Come take a break from studying, meet other students and enjoy some arts and crafts and insomnia cookies.
  • CIS Faculty and Postdoctoral Fellow Halloween TGIF – Quain Courtyard, October 29, 5-8pm
    There will be a Halloween Costume Contest with gift card prizes for winners! There will also be a pumpkin carving event, food, and an expanded selection of alcoholic and non-alcoholic drinks.
Categories
Students

Shirin Saeedi Bidokhti receives 2021 NSF CAREER Award

Shirin Saeedi Bidokhti (Illustration by Melissa Pappas, Courtesy of Penn Engineering Today)
This article originally appeared in Penn Engineering Today, written by Melissa Poppas.

Humans have never been more connected to one another, though the speed with which we can share with one another has its drawbacks. For example, the spread of COVID-19, as well as misinformation about it, have both been facilitated by our highly connected online and in-person networks. Fortunately, the branches of mathematics known as information theory and network theory can help us to understand how both systems work and how to control them.

NSF CAREER Award recipient Shirin Saeedi Bidokhti, Assistant Professor in Electrical and Systems Engineering, will use the grant to conduct research on both online social networks and COVID-19 contact tracing networks. As case studies, these real-word examples will inform networked systems’ theoretical foundations, as well as the design of learning and decision-making algorithms that help us to make sense of them. She will also use the funding to develop a new course module that brings information and network theory into practice for undergraduate students at Penn.

Using a combination of tools from information theory, network theory and machine learning, Saeedi Bidokhti aims to narrow the gap between theory and practice through algorithm-informed real-time data sampling, estimation and inference in networked systems. Her work aims to produce smarter algorithms that can extract information, infer about these systems, and ultimately provide more precise control of them.

While such algorithms are already improving our ability to understand complex networks, there is always a tradeoff that needs to be considered when it comes time to use that information.

“In information extraction, knowing when to sample with real-time data makes a difference, says Saeedi Bidokhti. “It helps us to know if we should act now or wait to sample, facing the tradeoff of gathering the most information while minimizing costs to most efficiently control the system.”

To read the full article, visit Penn Engineering Today.