Blog

Invisible Disease Awareness

“According to the Disabled World website, an estimated 10% of the U.S. population has what could be considered an ‘invisible’ disease, defined as a health condition that causes significant impairment and undermines the overall quality of life but does not outwardly manifest itself in ways that are apparent to others.” [1]

While normally our focus is Postgres, we wanted to take a moment to bring attention to the People side of People, Postgres, Data. May is mental health and Ehlers-Danlos awareness month. Ehlers-Danlos Syndrome (EDS) is a rare condition that affects the collagen throughout the entire body, resulting in dislocations, subluxations, lack of joint stability and support, tendinosis, and debilitating pain. There is no cure and the symptoms are life long.

In 2020, PostgresWarrior (AKA Amanda Nystrom) was diagnosed with Hypermobile EDS and Fibromyalgia. She is an instrumental and invaluable member of the People, Postgres, Data community. She has driven us forward in ways that so many of us never see and yet require to succeed. Many of our community are affected by invisible diseases - let's take a moment to appreciate what they accomplish and fight for in Postgres/Open Source.

Upcoming webinars | RSVP here

• May 25, 1pm ET: Creating a Resilient PostgreSQL Cluster with Kubegres

• June 15, 1pm ET: When it All Goes Wrong - Incident Response in Large Postgres Databases

• June 23, 1pm ET: Making Postgres Fly on Kubernetes

• June 29, 1pm ET: Implementing Cluster File Encryption in Postgres

Community Chat

Our Discord Channel has 2000 community members waiting to participate in your Postgres success. Join us today with a community that has rule #1 of: Be Nice.

Podcasts

• Building Your Data Warehouse On Top Of PostgreSQL - Episode 186
• More than a Refresh: Stephanie Kirmer, Data Scientist for Saturn Cloud

1. https://www.socialworktoday.com/archive/072417p32.shtml

With 2020 firmly in the rear view mirror, it is time to look forward and down the highway of 2021. The organizers of People, Postgres, Data have gathered over chat, email, phone, and even a few socially distant, in-person events to determine a strategy for continuing as the most influential and positive community for all things Postgres related.

Sad face

The goal is to resume in-person events. However, out of concern for the health and comfort of our global community, we have made the decision not to host any in-person events until Q4 of 2021. We are prepared to wait until 2022 if that is what the health officials recommend. We know that many will find this news disappointing and we are working diligently to ensure that the health and education of our community is the top priority.

Happy face

We are continuing our popular webinar series, adding new presenters with pertinent content for all of our attendees. We will be adding more professional development and data problem solving topics to our library, and we will no longer be limiting education to just Postgres, as many data and human problems are neutral in the particular platform we happen to enjoy. If there’s a topic you’d like to present or see, we’d love to hear from you!

RSVP for upcoming scheduled events

• January 26, 1pm ET: All we need to work with SQL is SQL
• January 27, 1pm ET: PostgreSQL Forks and Knives
• February 3, 2pm ET: Postgres for SQL Server Users
• February 4, 1pm ET: Configuring PostgreSQL for Faster Analytic Query Performance
• February 23, 1pm ET: Blockchain as a Database

Ecstasy 

Postgres Conference 2021: Digital will be happening in May of this year! An overwhelming feeling of great happiness and excitement has our dopamine pumping, and the whole People, Postgres, Data team is basking in it. 

As an all digital conference, we will offer a similar environment to what our community has come to expect: best in class content, professionalism, and top-tier educational opportunities for all who attend! Keep your eyes peeled over the next few weeks for more information on speaking opportunities and how to attend!

We all know that we should produce good documentation, but we’ve all skipped out on it every now and then. It’s often a lot of  mundane effort when you could be doing new exciting things.

Then days, weeks, months or even later the questions start coming in from your team-mates or new hires: why is this column like this, what does this column really describe etc. You no longer remember and you have to spend time to understand the database you’ve made.

One of the great things about PostgreSQL is that documenting your database is a piece of cake and that it saves you more time than you spend on adding comments. It’s without exaggeration less time consuming per table than answering a single question like the ones above.
Thankfully the syntax isn’t horrible and adding comments won’t be a slow affair if you have a lot of existing data unlike some other DB engines I’ve worked with.

PostgreSQL has support for storing comments directly in your schema:
You can add a comment to a table, a schema, an operator and so on.

Adding them is simple:

Some tools have better support for showing them than others and I hope to see that being improved in the future, for instance: Datagrip, my editor of choice will not show comments in the database explorer unless you hover over the object it does include it in the in-editor information pop-up when hovering over a column, function or table.

My personal favorite for generating visually pleasing documentation with little effort is SchemaSpy it requires Java and the JDBC driver to run and it generates pretty documentation in about a minute for a complex schema:

The output is in the form of a relatively pretty website like this : Sample

I would say that this is quite a few levels of quality above what most companies have for their internal databases and with a lot less effort. As you can see from the sample it even supports markdown.

If you have suggestions for even better tools for the job, please join the People, Postgres, Data: Discord server

This work is unrelated to my job at Bamboo Solutions AS

With the pending GA release of Star Link, more cities adopting municipal WIFI, and the growth of remote work due to the pandemic, one would assume that cell phone providers would be falling hand over fist to provide quality services at a quality price. Unfortunately this is not the case and it is hurting the future of our workforce.

Carriers currently think that 30GB of Hotspot data is enough for the average digital nomad. While that may have been the case previously, it isn’t any longer. There is a hundreds-of-thousands-strong workforce specializing in Information Technology, Digital Design, Web Development, and other creative industries. They are living, working and adventuring in RVs, Sprinters, Skooolies (pictured), and other vehicle dwelling options. This community is nomadic by design and should not be artificially limited by technology. 

Why is it that we are being charged so much money for so little? Outside of two providers announcing rural initiatives, we are still acting like data needs are small and that they come at a quality price. Data is ubiquitous; it is the new water (for getting work done). In the new economy workers need faster, higher quality, and unlimited amounts of data to get their job done.

Where could you go and what could you experience if you weren’t tethered by the faux limitations of bandwidth?

Resources

• Community Chat 
• Upcoming Free Webinars
• Community Blogs

Featured Content

• Using PostgreSQL, PostGIS, and pgRouting for street sweeping
• Zheap dropped by EnterpriseDB: Cybertec takes lead
• Professionalism Matters

By Magnus Brun Falch CoFounder of Bamboo Solutions AS

From zero to PostgreSQL extension in three hours.

I recently had a personal side-project which included setting up a working REST backend for a mobile and web application. There were numerous challenges to overcome but my available hours to work on them were limited. I had a few evenings to spare on this project and didn’t feel like writing a lot of boilerplate in C# to achieve it.

A trusted friend had praised PostgREST many times. My testing showed it outclassing ASP.NET Core for all my test cases by an order of magnitude, half a year earlier.
I finally had a chance to use it as the base for my API layer in one of my projects.

One thing that this project called for which would not normally be easy to achieve using PostgreSQL alone, is to pre sign requests for an S3 compatible storage platform. I had 6 hours left in the budget for this feature, after all the firm requirements were estimated.

I didn’t want to write a full implementation of the relevant parts of the S3 spec from scratch or using pgcrypto. My first instinct was to have a look at the different procedural languages available in PostgreSQL, some of which are Tcl, Java, JS, Python and Julia. None of them were too tempting at the moment.

Luckily I was an early member of the new People, Postgres, Data: Discord server and there are some very interesting and clever people there, Eric Ridge, the main author of ZomboDB (ZDB) among them.

I’ve kept an eye on ZDB for years after doing some research into ElasticSearch. My project included text search so I relished the opportunity to chat with him. Among our many topics of conversation were the upcoming release of a new Rust based version of ZDB and its underlying framework: PGX. I asked him about whether he thought PGX would be a good fit for my remaining challenge. He said “Absolutely”. I spent an hour setting up an Ubuntu VM with all the needed tooling & dependencies including building and testing the “hello world” example. A thorough reading of the man page would likely have saved me half an hour.

The next 30 minutes were spent identifying the most promising Crate to use. Within three hours of writing the first line of code I had a working plugin. About 80% of the time spent was lost to unfamiliarity with the language and writing it using nano over ssh instead of setting up a proper IDE like Jetbrains CLion. I also struggled a bit with finding the correct way of extracting the value from a result of an async function in a foreign language.

In total I spent 4 hours and 25 minutes from the time I started the fresh Ubuntu 20.04 server I was going to use as a Dev environment, to having a working build of an extension. It can be called inline in PostgreSQL to sign download requests in less than 1ms on my Intel I5 2500 testbed.

 I’ve had zero experience with Rust and have never read any teaching materials or looked at any tutorials for it before that evening. What I’d heard about Rust previously was that it’s really fast, and that the compiler would yell at you until your code is of acceptable quality.

Example of the PGX simplicity:

The extension was merely for internal use in a specific project and has some rough edges. It is published on GitHub and needs some more polish before I would consider it a production ready extension. Pull requests are welcome if you have suggestions before I get the time to work on it some more.

Notes:

• PGX is a framework for writing PostgreSQL extensions using Rust.
• ZomboDB integrates Elasticsearch and PostgreSQL.
• PostgREST consumes a PostgreSQL schema and generates API endpoints.

This work is unrelated to my job at Bamboo Solutions AS

What’s your view?

Over 25 years ago I got into an argument with my boss. He was frustrated with all the tasks that needed to be completed and feeling overwhelmed. I said, “Hey, why not go to a park and take in the scenery? You could work there too.” Those were the days before the Internet was required to complete your day to day. He was incredulous, “I don’t pay 1200.00 a month for an office to go work at a park.” It is funny how a single conversation can stick with you throughout life. Now that person is in the Cannabis industry and takes a completely different view on life.

Today I write this newsletter from Henry’s Lake, Idaho just about 30 minutes from Yellowstone National Park and the newsletter image is a photo I took this morning as I was about to make coffee. During these trying times with the pandemic and economic uncertainty it is vital that all of our community take a moment to reflect on what their view is. How is it that you are living your life? Are you putting people first? Are you helping each other as you move through the day? Are you in need of help? These are all questions we should ask ourselves and others every day.

With People, Postgres, Data we make an earnest effort to put people first, professionally and personally. This is why when we have physical events, we have tracks that are not traditional PostgreSQL faire including our Career Fair and Professional Development and Leadership tracks.It is also why we try to keep our digital events free. The better people we are, the better professionals we can be and the better the professional we are, the more we are able to help people. It is also why we started a new community chat server with Discord. Though the server is there to help you with Postgres, it also exists to embrace community with channels such as #watercooler, #food, #games and #professional-advice. It is a holistic approach that allows people to be people, not bytes.

• Community Help and Chat

RSVP for upcoming free digital Events

• September 29, 10AM PST: Introduction to PostgreSQL ColumnStore Indexes

• September 30, 10AM PT: Live Demo: Creating A Single Point Of Access To Multiple Postgres Servers Using Starburst Presto

• October 6, 9am PT: Data processing more than billion rows per second

• October 7, 10am PT: Database Isolation Levels, Data Issues and Global Transaction Consistency

• October 14, 10AM PT: Live Demo: Unlock Data In Postgres Servers To Query It With Other Data Sources LIke Hive, Kafka, Other DBMSs, And More.

• October 20, 10AM PT: PGX: Build Postgres Extensions with Rust

• October 21, 10am PT: Using PostgreSQL, PostGIS, and pgRouting for street sweeping

• October 27, 10AM PT: Logical Replication lessons learned for the Data Warehouse

• October 28, 10AM PT: How to build local communities: a meetup perspective

• November 10, 10AM PT: CYPEX: Revolutionizing PostgreSQL Application Development

• November 12th, 10am PT: Blockchain as a Database

As Postgres Conference pushes forward in the brave new world, we evolve and increase the ability for the People, Postgres, Data community to succeed. As a part of our positive adaptation we have a new website that features upcoming events, professional content (written and video), and the best in our written community via “Community Content”.

Discord

In an effort to provide a modern, friendly, and inclusive community platform, we have launched a Discord server for all things Postgres. We are providing a helpful experience with rule #1 being: Be Nice. Our discussions will branch out beyond the core of PostgreSQL and provide a forum for success with Postgres and related technologies. Join us for what is guaranteed to be a refreshing experience for the community: https://discord.gg/tjxNBCz

2021

We are seeking feedback from our community on 2021 in-person events. We are currently considering the East Coast event for October 2021 and the West Coast event for December 2021. Please help us in determining the type of event you would like to participate in!

Upcoming live events

We currently have the following webinars scheduled through October:

• September 9th, 10AM PST: PostgreSQL Parallelism Do’s and Don’ts
• September 15th, 10AM PT: Monitoring Strategy Considerations
• September 29th, 10AM PST: Introduction to PostgreSQL ColumnStore Indexes

• September 16, 10AM PT: How To Query Against Multiple Postgres Instances As If They're One Server Using Starburst Presto
• September 29, 10AM PST:Introduction to PostgreSQL ColumnStore Indexes
• September 30, 10AM PT: Live Demo: Creating A Single Point Of Access To Multiple Postgres Servers Using Starburst Presto
• October 14, 10AM PT: Live Demo: Live Demo: Unlock Data In Postgres Servers To Query It With Other Data Sources LIke Hive, Kafka, Other DBMSs, And More.

Like most conference organizers we are learning to adapt to the new world; a world where physical events are no longer viable (at least in 2020). A world where people are genuinely and realistically concerned that an in-person event would increase their chances of receiving or spreading a life threatening virus.

The question is: Are in-person events a thing of the past?

The answer to that question is a difficult one. Our friends at O'reilly and Associates have permanently canceled their in person events. Our friends in Europe recently canceled the well respected PgConf.EU and Ibiza. We had to cancel our 2020 marque event in NYC in March and our upcoming Silicon Valley conference. The local community organizer website Meetup.com has even modified their capabilities to allow for online meetups. 

Humans in general seek out fellow human contact. That contact is usually of reasonably like minded individuals or at least mutual interests. This is why events like Postgres Conference are successful, because irrespective of any personal beliefs we are all there to learn and enjoy fellow Postgres professionals. But are virtual meetups and conferences going to be enough to satisfy that connection or are people going to demand a return to a historical norm?

Challenges

Even before COVID-19, in-person events came with challenges that put significant pressure on volunteers. Between cultural communication differences, having an independent Code of Conduct committee, pricing, economies of scale, partner demands, and now social distancing, conferences are now going to be more complicated than ever. A room that once could comfortably seat 100 can now only properly sit 30. An exhibit hall is likely out of the question and one-on-one mentoring and networking are likely not going to be viable.  How do we work around these limitations? Is it worth it? Are the people in our community even interested anymore or is it time to accept a new norm?

Opportunity

Without question this is a time of reflection, continued development of relationships, and looking into the magic 8-ball; a continual asking of questions to find the right path forward. The pandemic is a tough foe but true leaders are looking forward and trying to find ways to continue to serve. For that to be successful we need your help. We have put together a poll (that can be found here) to gain insight into what opportunities we may be able to pursue in the future. Please take a couple minutes and help shape the future of Open Source events. 

As a closing, we are actively moving forward with Digital Events across the globe and have an unending Call for Presentations open for Webinars. If you have any feedback or brilliant ideas, please send them to us via organizers@postgresconf.org.

Blatant Poll Link 

At Postgres Conference we are always optimizing our People, Postgres, Data experience, and as more events move to Digital,  we have worked hard to perfect that collaborative experience for our community. The hard work of our volunteers has paid off, our content is top notch, and our participant account is consistently high.

With a now-proven track record, we are pleased to announce that we are accepting Community and Commercial presentations via the Postgres Webinar Series Call for Presentations through the end of 2020. If your presentation is accepted we will contact you directly to arrange an appropriate time to deliver your content.

Recent awesome news

KDE, arguably the most complete desktop experience for Linux, is now hosted on a private instance of Gitlab. As is proper, Gitlab only supports Postgres as the backend database.

For those who who love #FarmFresh or #farmtotable, there is a new website that showcases Farms that are delivering. It was developed using React, Rails, and Postgres. 

Upcoming Digital Events

• July 7, 10am PT: Deep Dive into PostgreSQL Indexing

A Deep Dive into PostgreSQL Indexing

• July 15, 10AM PT: Working with JSON Data in PostgreSQL vs. MongoDB

Working with JSON Data in PostgreSQL vs. MongoDB

Previous Digital Events:

• PostGIS Training series:
• Getting Started with PostGIS and OpenStreetMap by Ryan Lambert
• Advanced Operations and Features by Ryan Lambert
• Exploring OpenStreetMap from PostGIS:  Tools and Queries by Ryan Lambert
• Loading PostGIS with osm2pgsql by Ryan Lambert
• PostGIS Spatial Queries and Performance Tuning by Ryan Lambert
• PostgreSQL 12 and PostGIS 3.0 by Ryan Lambert
• Ops:
• FPGA: an Easier Path to Parallelism for Postgres? by Andrew Ellicott
• How to Deploy a SQL Database on Kubernetes by Chris Casano
• How to Move Data from Oracle to Postgres in Near-Real Time by Joshua Drake
• A Deep Dive into PostgreSQL Indexing by Ibrar Ahmed
• Mostly mistaken and ignored PostgreSQL parameters while optimizing a PostgreSQL database by Avinash Vallarapu
• Deeper Understanding of PostgreSQL Execution Plan: At plan time and run time by Jobin Augustine
• A Deep Dive into PostgreSQL Indexing by Ibrar Ahmed
• Enterprise Authentication and Connection Pooling by Erik Brandsberg
• Case Studies
• YugabyteDB: a distributed PostgreSQL database by Bryn Llewellyn
• Community vs. Enterprise Open Source – Which is Right for Your Business? by Justin Reock
• Postgres vs. MongoDB for real-time machine learning on wind turbine data by Luis Manuel Carril
• Turning PostgreSQL into a Cloud Native Data Service by Julian Fischer
• Scaling PostgreSQL: Never Shard Again and Gain Effortless Scale by Jim Walker
• Bring Compression to Postgres at Zero Cost of Performance by Tong Zhang

Postgres Conference organizers come to us through a variety of channels, but they all have their own unique story and pathways. Unlike many of our organizers, Lindsay Hooper’s background is non-technical and focused on the logistics, partnerships, and outcomes of Postgres Conference. Read on to learn more about her and her non-profit organization of choice

What is your background?

I majored in art and film at Bucknell University, and I started my career working in art galleries in Manhattan. I quickly moved to events and marketing, and within a few years I started doing events and marketing for tech companies and startups.

What do you do in your full-time job?

I have my own events company called LRH Events, and most of my clients are either in the non-profit or tech worlds. Events are a great tool for gaining brand awareness and for creating community, but getting an event off the ground takes some serious marketing chops, and so I do marketing as well.

What’s your favorite part of Postgres Conference?

There’s so much to love about Postgres Conference. When it comes to the conference itself, I love the logistics and planning that goes into what the event actually looks like. I love everything from brainstorming what the conference should look and feel like, to coordinating with speakers, streamlining talks, and coming up with schedules.

Beyond the actual events, my favorite part is the People, Postgres, Data community that surrounds the organization. I’ve never experienced such a truly people-first group, and I think that that shows in the fact that this isn’t a twice a year conference - it’s a constant connection that’s focused on people, not just Postgres or open source technology.

How do you spend your free time?

I am based in NYC, which affords me the opportunity to spend a lot of my free time exploring the city’s restaurants and museums. I highly recommend an afternoon at The Met, followed by dinner down in the East Village at either Raclette or Dirt Candy. 

Beyond that, I’ve been involved with a non-profit called Mouse for the last five years, so I also spend a few evenings each week working on fundraising and awareness initiatives. Mouse’s mission is to provide vulnerable youth with the computer science skills needed to enter and succeed in higher education and the high tech workforce. We are committed to fostering greater diversity and humanity in STEM and empowering youth - and all those that educate them - to access and amplify technology as a force for good.

How did you begin working with Mouse and what’s your role there?

I started working with Mouse five years ago when they launched the Diversity in Tech Awards, which is their annual awards event that honors folks in the tech ecosystem who foster greater diversity and humanity in STEM and empower youth. They needed event support, and I was able to step in to help.

Since then, I joined Mouse’s associate board, became the VP, and am currently the Associate Board President. 

What are the biggest challenges facing STEM education right now?

The greatest challenges facing education as a whole have shifted in the last few months due to the COVID-19 pandemic. In speaking with Mouse students, I’ve learned that a lot of what they’re up against is the variety of platforms that they’re receiving school work on and the lack of collaboration. On the flip side, most teachers are experiencing a learning curve themselves when it comes to remote teaching.

During the COVID-19 crisis, Mouse has played a frontline role in meeting the emergency educational needs. Responding to a call from the NYC Department of Education after schools closed, Mouse has:

• Trained more than 4,000 teachers in online teaching methods so they can reach their students.
• Helped more than 100,000 NYC students get access to online learning
• Digitized youth programs like the Emoti-Con Virtual Showcase and Mouse Design League for students to continue their computing projects
• Made our STEM and computer science learning platform, Mouse Create, free for all users during the crisis

How can I get involved in Mouse?

If you’re interested in getting involved with Mouse, please feel free to reach out to me directly at Lindsay@postgresconf.org

Alternatively, Mouse is proud to be able to continue our work with students and educators during the COVID-19 crisis. Please give generously today at www.mouse.org/donate so we can continue to make great strides in finding new ways to help our community in these challenging times.